Citation |

- Permanent Link:
- http://ufdc.ufl.edu/AA00024495/00001
## Material Information- Title:
- Matched-filterbank approaches to spectral analysis and parameter estimation
- Creator:
- Li, Hongbin, 1970-
- Publication Date:
- 1999
- Language:
- English
- Physical Description:
- xii, 122 leaves : ; 29 cm.
## Subjects- Subjects / Keywords:
- Apes ( jstor )
Capons ( jstor ) Covariance ( jstor ) Estimation bias ( jstor ) Estimation methods ( jstor ) Estimators ( jstor ) Signals ( jstor ) Sine waves ( jstor ) Statistical discrepancies ( jstor ) Supernova remnants ( jstor ) - Genre:
- bibliography ( marcgt )
non-fiction ( marcgt )
## Notes- Thesis:
- Thesis (Ph. D.)--University of Florida, 1999.
- Bibliography:
- Includes bibliographical references (leaves 116-121).
- General Note:
- Typescript.
- General Note:
- Vita.
- Statement of Responsibility:
- by Hongbin Li.
## Record Information- Source Institution:
- University of Florida
- Rights Management:
- The University of Florida George A. Smathers Libraries respect the intellectual property rights of others and do not claim any copyright interest in this item. This item may be protected by copyright but is made available here under a claim of fair use (17 U.S.C. Â§107) for non-profit research and educational purposes. Users of this work have responsibility for determining copyright status prior to reusing, publishing or reproducing this item for purposes other than what is allowed by fair use or other copyright exemptions. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder. The Smathers Libraries would like to learn more about this item and invite individuals or organizations to contact the RDS coordinator (ufdissertations@uflib.ufl.edu) with any additional information they can provide.
- Resource Identifier:
- 41928665 ( OCLC )
ocm41928665
## UFDC Membership |

Downloads |

## This item has the following downloads: |

Full Text |

MATCHED-FILTERBANK APPROACHES TO SPECTRAL ANALYSIS AND PARAMETER ESTIMATION By HONGBIN LI A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY. UNIVERSITY OF FLORIDA 1999 This work is dedicated to my wife, Hong. ACKNOWLEDGEMENTS I would like to express my sincere gratitude to my advisor, Dr. Jian Li, for her constant support, encouragement, enthusiasm, and patience in guiding this research. My deepest appreciation goes to Dr. Petre Stoica for his numerous insightful remarks and suggestions which comprehensively influenced this work. Special thanks are due to Drs. Jose C. Principe, Fred J. Taylor, William W. Edmonson, and David C. Wilson for serving on my supervisory committee and for their contribution to my graduate education at the University of Florida. I also wish to thank Zhaoqiang Bi, Robert Stanfill, and other fellow graduate students with whom I had the great pleasure of interacting. Drs. Zheng-She Liu and Guoqing Liu have my gratitude for sharing many interesting discussions with me. I would like to gratefully acknowledge all the people who helped me during my Ph.D. program. iii TABLE OF CONTENTS ACKNOWLEDGEMENTS ............................ iii LIST OF TABLES ................................. vii LIST OF FIGURES ................................ viii A BSTRACT . . . . . . . . . . . . . . . . . . xi CHAPTERS 1 INTRODUCTION .................. .......... 1 1.1 Background and Scope of the Work ................. 1 1.1.1 Capon, APES, and MAFI Spectral Estimators ...... 1 1.1.2 Efficient Implementation .................... 4 1.1.3 Amplitude Estimation ................... 5 1.1.4 Covariance Sequence Estimation ............... 5 1.2 Significance and Contributions .................. 6 1.3 Organization of the Dissertation ..................... 7 2 LITERATURE SURVEY ................. ....... 9 2.1 Filterbank Approaches and Capon Estimator .......... 9 2.2 Covariance Estimation ................. ..... 13 3 MAFI APPROACH TO SPECTRAL ESTIMATION .......... 16 3.1 Forward and Backward Data Vectors ............... 16 3.2 MAFI Filters....... ............. ....... 19 3.2.1 Capon Filter ....... .... ......... 20 3.2.2 APES Filter ........ .... .. ...... 21 3.2.3 Another Matched Filter .................. 22 3.3 Analyses of MAFI Approaches .. .................. 24 3.3.1 Computational Complexity ........ ......... 24 3.3.2 Statistical Performance .............. 25 3.4 2-D Extensions .......... .. ............. 27 3.5 Numerical Examples .............. ...... .. 31 3.5.1 1-D Complex Spectral Estimation .... .... .... 31 3.5.2 2-D Complex Spectral Estimation ......... . 33 3.6 Summary ................. ............ 36 iv 4 EFFICIENT IMPLEMENTATION OF CAPON AND APES ...... 39 4.1 Introduction .......... ... .. ....... 39 4.2 Efficient Implementation of APES .................. 39 4.3 Extension to Capon . ....... .................. 43 4.4 Numerical and Experimental Examples ............ 44 4.5 Summary ........... ...... ............. 45 5 AMPLITUDE ESTIMATION ................... ... 47 5.1 Introduction .......... .. ........... 47 5.2 LS Amplitude Estimators ......... ...... ... . 50 5.2.1 LSE(1, 0, K ) .. .... .. .. . .. .. . . 50 5.2.2 LSE (1,0,1) .......... ... .. ....... 52 5.3 WLS Amplitude Estimators ................... .. 54 5.3.1 W LSE(L, 0, K) ........... .. ... ...... .. 54 5.3.2 WLSE(L, 0,1) ........................57 5.4 MAFI Amplitude Estimators ................... 58 5.5 Numerical Examples ......................... 62 5.5.1 Estimation Performance versus SNR ........... 63 5.5.2 The Effect of M ......... ............ .. 68 5.6 Application to System Identification ............... 68 5.6.1 System Identification Using Amplitude Estimation . . 70 5.6.2 Numerical Examples . . . .... ....... 73 5.7 Summary ................... . ............ 78 6 CAPON ESTIMATION OF COVARIANCE SEQUENCES ...... 80 6.1 Introduction . . . . . . . . . . . . . . 80 6.2 Standard Covariance Estimator and Outlook .......... 82 6.3 Capon PSD Estimator ........................ 84 6.4 Capon Covariance Estimator ................... 87 6.4.1 Exact Method ................. ...... 87 6.4.2 Approximate Method ................... 89 6.4.3 Computational Aspects .................. 90 6.5 Numerical Results ................... ....... 91 6.5.1 ARMA Covariance Estimation .............. 92 6.5.2 AR Coefficient Estimation for ARMA Signals ....... 97 6.5.3 MA Model Order Determination ............. 98 6.6 Sum m ary . . . . . . . . . . . . . . .. 100 7 CONCLUSIONS ............................ 102 7.1 Summary Remarks ..... ................... .. 102 7.2 Future Work ................. .......... 105 APPENDIXES A PROOF OF (3.42) AND (3.43) ................ ...... 106 B PROOF OF THEOREM 3.3.1 ...... ................ 108 V C PROOF OF THEOREM 3.3.2 ..................... 110 D PROOF OF THEOREM 6.4.1 ......... .............. 114 REFERENCES ............................. 116 BIOGRAPHICAL SKETCH ................... ........ 122 vi LIST OF TABLES 5.1 Choice of M for the WLS and MAFI amplitude estimators ...... 70 6.1 Comparison of the computational burdens of the standard and Capon methods with N = 256 and N = 512. .................. 91 6.2 The ARMA processes used in the numerical simulations. ....... 92 vii LIST OF FIGURES 1.1 Synthetic Aperture Radar (SAR) images of a simulated MIG-25 airplane obtained by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES ......................... 3 3.1 The I-D complex amplitude of the sum of 15 sinusoids used in the simulations. (a) Real part; (b) Imaginary part. ........ . 33 3.2 Empirical bias and variance of the 1-D MAFI estimators as SNR1 varies when N = 64 and M = 15. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated am plitude . . . . . . . . . . . . . . . . . 34 3.3 Empirical bias and variance of the 1-D MAFI estimators as the filter length, M, varies when N = 64 and SNR1 = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. ........... ......... ....... 35 3.4 Empirical bias and variance of the 2-D MAFI estimators as the SNR1 varies when N1 = N2 = 32 and M1 = M2 = 8. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. .............. ........... ..37 viii 3.5 Empirical bias and variance of the 2-D MAFI estimators as the filter length, M = M = M2, varies when N1 = N2 = 32, and SNR1 = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. ......... . 38 4.1 SAR images obtained from the ERIM data by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES. ....... 46 5.1 PSD of the test data that consist of three sinusoids and an AR(1) noise process .................................. 63 5.2 Empirical MSEs and the CRB versus local SNR when N = 32, M = 8, and the observation noise is colored (an AR(1) process). (a) a3. (b) a,. 65 5.3 Empirical MSEs and the CRB versus local SNR when N = 32, M = 8, and the observation noise is white. (a) a3. (b) al. ........ 67 5.4 Empirical MSEs and the CRB versus M when N = 32 and the observation noise is colored (an AR(1) process with a2 = 0.001). (a) a3. (b) a ................. ................ 69 5.5 Averaged RMSEs and the number of flops versus N for the first system when the observation noise is white (a2 = 0.01) and M = 20 for APES1 and MAFI1. (a) RMSE of a-parameters. (b) RMSE of b-parameters. (c) Number of flops. ................... ........ 76 5.6 PSD estimate of the output of the first system corrupted by white noise with a2 = 0.01 and N = 200. ................... 77 5.7 Averaged RMSEs and the number of flops versus N for the second system when the observation noise is colored (an AR(1) process with a2 = 0.01) and M = 20 for APES1 and MAFI1. (a) RMSE of aparameters. (b) RMSE of b-parameters. (c) Number of flops ... 78 ix 6.1 Power spectral density estimates by using Capon-i and Capon-2. The plots are the averages of 100 independent realizations. (a) N = 256, M = 50; (b) N = 32,M = 10. ....... ..... ... ........ .. 86 6.2 Pole-zero diagrams for ARMA test cases. (a) ARMA1; (b) ARMA2; (c) ARMA3; (d) ARMA4..... ........ ............ ..93 6.3 True power spectral densities. (a) ARMAl; (b) ARMA2; (c) ARMA3; (d) ARMA4 ............... .. ........... 94 6.4 True covariance sequences. (a) ARMA1; (b) ARMA2; (c) ARMA3; (d) ARM A4...................... ............ 95 6.5 Covariance sequence estimation with N = 256 and M = 50. The mean-squared errors (MSEs) of the covariance estimates, normalized with respect to r(0), are based on 100 independent realizations. (a) ARMA1; (b) ARMA2; (c) ARMA3; (d) ARMA4 ............ 96 6.6 The AR coefficient estimation of the ARMA signals via the overdetermined modified Yule-Walker method with N = 256 and M = 32. The curves are the summations of the mean-squared errors (MSE) of all the AR coefficient estimates versus the numbers of included equations, which have been set as 4, 8, 16, 32, 64 and 128, respectively. The MSE curves are based on 100 independent realizations. (a) ARMA1; (b) ARMA2; (c) ARMA3; (d) ARMA4 .................. 99 6.7 10 superimposed realizations of the MA covariance sequence estimates with N = 64 and M = 6. (a) The standard method; (b) The Capon method. ...... ...... .... ............... 100 x Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy. MATCHED-FILTERBANK APPROACHES TO SPECTRAL ANALYSIS AND PARAMETER ESTIMATION By Hongbin Li May 1999 Chairman: Jian Li Major Department: Electrical and Computer Engineering MAtched-FIlterbank (MAFI) estimators represent a general class of methods that make use of a set of matched filters for various estimation purposes. This dissertation investigates using MAFI approaches for complex spectral analysis, amplitude estimation for sinusoidal signals, and covariance sequence estimation. For complex spectral analysis, we show that the widely used Capon and the recently introduced APES estimators are both members of the MAFI approach, though neither was originally derived in the MAFI framework. We prove that, to within a second-order approximation, Capon is biased downward whereas APES is unbiased, and that the bias of the forward-backward Capon is one half that of the forward-only Capon. We also show that Capon and APES are of similar computational complexities and both are more involved than most Fourier-based methods, especially for 2-Dimensional (2-D) data. Efficient implementation schemes which substantially xi reduce the computational requirement are presented for the Capon and APES estimators. For amplitude estimation, we describe a large number of estimators which can be categorized as the Least Squares (LS), Weighted Least Squares (WLS), and MAFI methods. While all these methods are asymptotically statistically efficient, their performances in finite length of data samples are quite different. Specifically, we show that the WLS and MAFI methods outperform the LS methods whenever the observation noise is colored; we also show that the MAFI approach is very general and includes the WLS approach as a special case. For covariance sequence estimation, we present a Capon covariance estimator by Fourier inverting the Capon Power Spectral Density (PSD) estimates. We describe the Fourier inversion in both an exact and approximate ways, of which the latter is computationally more attractive but with some minor accuracy loss. We show that the Capon covariance sequence estimates are in general better than the widely used sample covariance sequences. xii CHAPTER 1 INTRODUCTION This dissertation is concerned with spectral analysis and parameter estimation using MAtched-FIlterbank (MAFI) approaches. This chapter serves as a general introduction to the background and scope of the work. Significance and contributions are also highlighted in this chapter. 1.1 Background and Scope of the Work 1.1.1 Capon, APES, and MAFI Spectral Estimators Spectral estimation is an important data analysis tool which has found applications in many diverse fields including speech analysis, telecommunications, radar and sonar systems, biomedical and seismic signal processing, and economics [75]. There are two broad classes of approaches to spectral analysis: non-parametric approaches and parametric approaches. While non-parametric methods typically postulate no model for the observed data, parametric approaches do assume some model so that the spectrum is represented by a set of parameters, thereby reducing the spectral estimation problem to that of estimating the parameters of the model. Parametric methods may offer more accurate spectral estimates than non-parametric methods, provided that the data indeed observe the model assumed by the former. In the more likely cases where an accurate data model is not available, parametric methods will be sensitive to model mismatch; hence using the robust non-parametric approaches in such cases may be a better choice. Recent studies in the literature show that there has been renewed interest in non-parametric approaches to spectral estimation. 1 2 Filterbank approaches to spectral estimation belong to the class of nonparametric spectral estimators. An important member of filterbank approaches is the Capon spectral estimator [16] [18]. Unlike the classical Fourier-based methods which are data-independent, the Capon spectral estimator adapts to the processed data in a manner so that the noise components of the data are rejected substantially. During the past few decades, the Capon spectral estimator has been widely used because its higher resolution and lower sidelobes give it an advantage over the Fourier-based methods. Additionally, its robustness and less variability make it preferable to the parametric methods [75] [41]. In a recent study, it was empirically observed that using the Capon estimator for complex spectral estimation gives biased spectral estimates whereas the newly introduced APES (Amplitude and Phase EStimation) method appears to be unbiased [52] (see Figures 1.1(a) to 1.1(d) for performance differences of using the Fourier-based methods, Capon, and APES for spectral estimation). The fact that both Capon and APES make use of a set of finite impulse response (FIR) matched filters was observed in [70]. A number of results on the performance differences of the Capon and APES estimators were also reported therein. However, the study in [70] was somewhat limited since it only considered the forward-only (FO) Capon and APES estimators. Owing to the general belief that forward-backward (FB) approaches to spectral estimation usually provide more accurate results and are used more often than their FO counterparts, it is of interest to conduct a study that investigates how the FB Capon and APES spectral estimators perform when compared with one another as well as with their FO counterparts. In some sense, filterbank approaches reduce the problem of spectral estimation to a filter design problem subject to some constraints [46]. In Section 3.2, we discuss a general rule for choosing the impulse responses of the filterbank so that the signal-tonoise ratios (SNRs) at the outputs of the filterbank are maximized. It follows that the 3 (a) (b) (c) (d) Figure 1.1: Synthetic Aperture Radar (SAR) images of a simulated MIG-25 airplane obtained by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES. 4 resulting filters are matched filters and we refer to this class of approaches to spectral estimation as MAtched FIlterbank (MAFI) approaches. We show that both Capon and APES are members of MAFI approaches, although none of them was originally derived in the MAFI framework (see [16] [52] for their original derivations). MAFI approaches to spectral estimation may also be used to devise new spectral estimators. Even though we show in Section 3.2.3 that a reasonable implementation of a seemingly novel MAFI spectral estimator reduces back to APES as well, it remains an open issue whether other interesting MAFI spectral estimators exist. The MAFI interpretation also provides insights into the Capon and APES estimators and the relationship between them. Specifically, in the framework of MAFI approaches, we show by means of a higher-order expansion technique in Section 3.3 that the 1-D (one-dimensional) Capon estimator indeed underestimates the true complex spectrum while the 1-D APES is unbiased (to within a second-order approximation); we also show that the bias of the FB Capon is one half that of the FO Capon. Furthermore, we show that these results can be extended to 2-D (two-dimensional) Capon and APES estimators. 1.1.2 Efficient Implementation For 2-D applications, a major concern of using Capon or APES is their computational loads which in general are much heavier than those of the traditional Fourier-based approaches. For a SAR image of size 128 x 128 formed from a 32 x 32 data matrix, as those in Figure 1.1, the number of flops required by Capon and APES implemented in the conventional way are 2.3 x 104 and, respectively, 3.0 x 104 times that of those by the Fourier-based methods. It should be mentioned that a SAR image of 128 x 128 is relatively small. As the size of the image increases, the amount of computation by Capon or APES increases drastically. In Chapter 4, we address the issue of how to efficiently implement Capon and APES for spectral estimation. 5 1.1.3 Amplitude Estimation Another problem related to complex spectral estimation is amplitude estimation for sinusoidal signals where it is assumed that the number and frequencies of the sinusoids are known a priori. The observation noise is stationary and maybe colored. In the case that the noise can be modeled exactly, the Maximum Likelihood (ML) methodology can be used and the ML amplitude estimates are statistically efficient. However, an exact model of the observation noise is usually not available and, moreover, ML methods are in general very sensitive to inaccurate model information. As such it may be a better choice to use methods that do not model the noise exactly. In Chapter 5, we describe a relatively large number of amplitude estimators which assume no model except for stationarity for the observation noise. The amplitude estimators can be categorized as three general classes, namely Least Squares (LS), Weighted Least Squares (WLS), and MAFI approaches to amplitude estimation, which are all asymptotically statistically efficient. For finite length of data samples, however, their behaviors are quite different. We show that under certain circumstances, the MAFI approach to amplitude estimation is equivalent to the WLS approach, and yet the former is more general and includes the latter as a special case. We also show that the MAFI and WLS methods in general give more accurate amplitude estimates than the LS methods. 1.1.4 Covariance Sequence Estimation The fact that covariance (the terms covariance and autocorrelation are used interchangeably with one another in this study) function and Power Spectral Density (PSD) are a Fourier transform pair makes the problem of covariance sequence estimation a research topic that is closely related to spectral estimation. Sample covariance sequences have been widely used in signal processing because of its computational simplicity and its consistency supposing that the given signals are ergodic 6 to the second order. When only a finite number of samples are available, using sample covariance sequences implies that the data beyond the observed duration either is zero or repeats itself periodically, which is certainly not a realistic assumption. There have been several attempts in the literature to derive other covariance estimators (see Chapter 2 for some details). It is known that the sample covariance sequence and the data periodogram constitute a Fourier transform pair. It is also known that the periodogram is a statistically inefficient (in fact inconsistent) PSD estimator [75]. This observation suggests that better covariance estimators may be obtained by Fourier inverting better PSD estimators. Since the MAFI PSD estimates are in general more accurate than periodograms, we propose a new covariance sequence estimator by Fourier inverting the MAFI PSD estimates. Specifically, we make use of the Capon PSD and such an approach is referred to as the Capon covariance sequence estimator. The same methodology can be similarly applied to the APES PSD estimates, though the APES covariance estimates are usually similar to the Capon covariance estimates. The reason is that, despite their different performance for complex spectral estimation, the Capon and APES PSD estimators usually behave quite similarly, especially for continuous spectra (also see the discussions in Section 3.3.2). It is observed that Capon covariance estimates are in general better than sample covariance estimates in terms of mean-squared errors (MSEs). 1.2 Significance and Contributions The main results of this dissertation are as follows. 1. We introduce a new general class of MAFI approaches to spectral estimation. We show that the Capon and APES estimators, though originally not derived in the MAFI framework, are both members of the MAFI class. 2. To within a second-order approximation, we prove that the Capon estimator gives biased (downward) complex spectral estimates whereas the APES method 7 is unbiased; we also prove that the bias for the FB Capon is one half that of the FO Capon. These theoretical results, supplemented with the empirical observation that Capon usually underestimates the spectrum in samples of practical length while APES is nearly unbiased, are believed to provide compelling reasons for preferring APES to Capon in most practical applications. 3. We present efficient implementation techniques for the MAFI spectral estimators. We show that by using such techniques the amount of computation required by the Capon or APES estimator is significantly reduced. 4. The MAFI idea is also extended for amplitude estimation for sinusoidal signals in colored noise. Specifically, we make extensions to the Capon and APES algorithms to multiple sinusoids with known frequencies. Furthermore, we describe a generalized MAFI approach to amplitude estimation for multiple sinusoids. 5. A new covariance sequence estimator is presented by Fourier inverting the Capon spectral estimates. The Capon covariance sequence estimates are shown to be more accurate than the conventional sample covariance sequence estimates. 1.3 Organization of the Dissertation The dissertation is organized as follows. Chapter 2 gives a literature survey of such topics as filterbank approaches to spectral analysis, the Capon spectral estimator and its applications, and covariance estimation. In Chapter 3, we first introduce the MAFI approach to complex spectral estimation. We next show that the Capon and APES estimators are both members of the MAFI approach, followed by computational and statistical analyses of the MAFI spectral estimators. Extensions to the 2-D case are also included. Chapter 4 addresses the implementation issue of the MAFI spectral estimators. Chapter 5 discusses amplitude estimation for sinusoidal 8 signals from observation corrupted by colored noise. The MAFI concept introduced in Chapter 3 is extended for amplitude estimation. In Chapter 6, we investigate covariance sequence estimation using the Capon spectral estimates. Finally, we summarize this work and outline future work in Chapter 7. CHAPTER 2 LITERATURE SURVEY Historical and modern perspectives on the general topics of spectral analysis and parameter estimation have been well documented in the literature [42] [62]. Many classical articles, both theoretical and application-oriented, have been reprinted [20] [43]. Excellent texts are also available [75] [41] [55]. In this chapter, we give a brief review of a number of subjects that are related to our work, namely filterbank approaches to spectral estimation, the Capon method and applications, and covariance estimation. 2.1 Filterbank Approaches and Capon Estimator Unless the observed signal can be modeled with a finite number of parameters, estimating the spectrum of a signal based on a finite length of observations is an ill-posed problem from a statistical standpoint, since we are required to estimate an infinite number of independent spectral values based on a finite number of samples. An assumption made by filterbank approaches and most other non-parametric methods is to assume that the PSD of the observed signal is (nearly) constant over a narrowband around any given frequency. Naturally one can proceed by passing the observed signal through a bandpass filter, which is swept through the frequency band of interest, and estimating the complex amplitude if complex spectral estimation is of interest (or measuring the filter output power and dividing it by the filter bandwidth if PSD estimation is the desired goal), a procedure adopted by all filterbank approaches. Obviously, how to choose the narrowband filters is a critical issue of filterbank approaches. Even though some of the classical Fourier-based methods, 9 10 such as periodogram by Schuster [67] and its various variations including the famous Blackman-Tuckey's method [5], can also be cast in the framework of filterbank approaches, originally they were not designed in such a manner. In other words, those methods made no attempt to purposely design a good bandpass filter to achieve some desired characteristics. A notable example of filterbank approaches to spectral estimation is the REFIL (REfined FILter) method which was first introduced in [77] and was further developed in [56] (also discussed in [8] [59] [61]). The REFIL method is close in spirit to the Daniell approach [22] to reducing the variance of the periodogram. That is, REFIL does not split the available samples in shorter stretches. The REFIL idea is to design a bank of filters which pass the signal components within the passbands as much as possible relative the total power and, in the mean time, attenuate the frequencies outside the corresponding passbands. It turns out that REFIL in general gives better spectral estimates than the traditional Fourier-based methods. Nevertheless, the REFIL filters share a common characteristic with the Fourier-based ones: they are all data-independent in the sense that they do not adapt to the received data in any way. Presumably, it would be beneficial to take the data properties into account when designing filterbanks. The Capon method is one (and perhaps the most famous one) of such datadependent filterbank spectral estimators. The Capon estimator was first named as the Maximum Likelihood Method (MLM) due to the the ML and Gaussian process context used in Capon's original work [16]. It turns out that MLM is a misnomer since it is not an ML spectral estimator and it does not possess any of the properties of an ML estimator. In [41] and [55], Capon's method was referred to as the Minimum Variance Spectral Estimator (MVSE) because it is derived by minimizing the variance of the output of a narrowband filter. Even the name MVSE is inaccurate in the sense 11 that Capon spectral estimates do not possess the minimum variance property. In this study it is simply named the Capon spectral estimator. The Capon spectral estimator was originally introduced by Capon [16] [18] for applications in multi-dimensional seismic array frequency-wavenumber analysis. It is reformulated by Lacoss [44] for applications to 1-D time-series problems. The constraints adopted by the Capon method are such that the signal at the current frequency is passed undistorted (with unit gain) while the output power of the overall frequency domain is minimized. A number of researches have been conducted to study the Capon spectral estimator. Specifically, Capon and Goodman demonstrated [19] [17] that the Capon spectrum has a mean and a variance that behaves like the averaged periodogram; that is, Capon spectra are usually less variable than periodograms. Additionally, Lacoss's empirical study [44] suggested that the resolution of the Capon method is between that of Burg's AutoRegressive (AR) spectral estimator [9] [12] and that of the periodogram. Lacoss's study also suggested that the statistical variability of Capon is less than that of the AR estimator. Burg later proved that the reciprocal of the Capon spectrum of order M, the length of the Capon filter, is equal to the average of the reciprocals of the AR spectra from order 0 to M [11]. Such an averaging effect explains the empirical observations made by Lacoss. Further researches in quantizing the resolution properties of Capon spectra were reported in [21] [58]. Since Capon's method is a basic spectral estimator, it may be used in any application where the spectrum of the studied signal plays an important role. Indeed, ever since its first appearance, the Capon spectral estimator has been widely used in many areas including radar, sonar, communications, imaging, geophysical exploration, astrophysical exploration, and biomedical engineering (see [35] [79] and the references therein). Rather than making a futile attempt at making a full document 12 in great length, we describe in the following a few typical applications of the Capon estimator. An interesting application of Capon's method is beamforming. Beamforming is used in conjunction with an array of sensors to provide a versatile form of spatial filtering. The objective of beamforming is to estimate the signal arriving from a desired direction in the presence of noise and interfering signals. If the desired and interfering signals occupy the same (temporal) frequency band, then temporal filtering cannot be applied to distinguish signal from interference. However, since the desired signal and interfering signals typically originate from different locations, such spatial diversity can be exploited to separate signal from interference using a spatial filter. In 1972, Frost made use of a linear constrained optimization technique and introduced an adaptive beamformer [27], referred to as the LCMV (Linearly Constrained Minimum Variance) beamformer in the array signal processing community. The basic idea of the LCMV beamforming is to constrain the response of the beamformer so that signals from the direction of interest are passed with specified gain while minimizing the output power due to interfering signals and noise arriving from other directions. One would immediately notice the similarity to the constraints adopted by Capon. Indeed, the LCMV beamformer is a direct extension of the temporal Capon filter to the spatial domain. Among the so-called statistically optimum beamformers, LCMV is perhaps the most popular one since it needs no auxiliary channels as required by the Multiple Sidelobe Canceller (MSC) [2], and, unlike the class of optimum beamformers proposed by Widrow et al. [83] which require reference signals, it is blind. A useful structure for LCMV implementation is the Generalized Sidelobe Canceller (GSC) [32]. GSC represents an alternative formulation of the LCML beamformer which changes the constrained optimization problem of LCML to an unconstrained one. The unconstrained nature lends GSC to adaptive implementation more readily than the original LCMV beamformer and hence GSC is the one used more often 13 in practise. GSC also found applications in direct-sequence Code-Division MultipleAccess (CDMA) communication systems for blind multiuser detection [36] [66] [65]. An important extension of the LCML beamformer was made in [7] [6] for applications in airborne radar systems. A typical signal environment faced by such systems consists of strong clutter/interference of complicated angle-Doppler spectrum which is unknown and may be varying in both time and space. The extension made by Brennan el al is to simultaneously combine the signals received on multiple elements of an antenna array (the spatial domain) and from multiple pulse repetition periods (the temporal domain). Such a technique is referred to as space-time adaptive processing that has received much research interest recently [80] [82]. Another interesting application of the Capon spectral estimator is SAR imaging [57] [3]. Conventional SAR imaging techniques are the FFT (Fast Fourier Transform) or windowed-FFT methods. A number of parametric spectral estimation methods have also been used for SAR imaging [26] [38] [33], though their interest in SAR imaging is limited because of their sensitivity to model errors. A comparative study in [24] showed that adaptive filterbank approaches such as Capon offers good SAR images and enjoys the advantage of robustness as compared to parametric methods. A number of modified Capon methods have also been suggested for SAR imaging [25] [4]. 2.2 Covariance Estimation One type of the covariance estimation problems is to estimate the covariance sequence from a finite number of data samples. A standard technique for estimating covariance sequences uses the biased or unbiased sample covariance estimator. The biased covariance estimator is more commonly used because it provides smaller MSEs than the unbiased one and guarantees the covariance estimates to be non-negative [55]. The problem of the sample covariance estimators is the unrealistic windowing 14 they assume on the observed data. By exploiting the AR spectral estimator, Burg proposed a technique that can offer covariance sequence estimates for any desired lag and hence avoids the windowing problem suffered by the sample covariance estimators [10]. However, it was shown in [78] that the Burg covariance estimates are less accurate and more variable than the sample covariance estimates. In Chapter 6, we describe a new method for covariance sequence estimation based on Capon spectra. Estimating structured covariance matrix of the observed data vectors represents another type of the covariance estimation problem. For example, the covariance matrix of a stationary complex signal is Hermitian and Toeplitz. However, the sample covariance matrix obtained from a finite number of data samples seldom has this structure. Structured covariance matrix estimation is of importance in a variety of applications including array signal processing and time series analysis [28]. An intuitive way to obtain structured covariance estimates is to force the desired structures on the sample covariance matrix, a methodology adopted by the Iterated Toeplitz Approximation Method (ITAM) [81] [15] [84]. Specifically, ITAM alternatively makes use of rank approximation (via singular value decomposition) and Toeplitzation along the diagonals until convergence is reached. Obviously, such a method is by nature heuristic and no optimality can be associated with it, though the ITAM covariance matrix estimate is in general closer to the true covariance matrix than the sample covariance matrix in the Frobenius norm sense. Optimum structured covariance matrix estimate may be obtained by maximizing the corresponding likelihood function as considered in [1] [13] [23] [29] [85]. However, since there exists no closed-form solution to the complicated nonlinear ML estimation problem for Hermitian Toeplitz matrices, the ML methods proposed in these studies are iterative and computationally involved, and, moreover, they are not guaranteed to yield the global optimal solution, which to some degree limits the interest in using the ML structured covariance matrix estimate in practical applications. An approximate ML method that makes use of the 15 Extended Invariance Principle (EXIP) [76] was recently presented in [49] [50]. This method provides asymptotic (for large samples) ML estimation for structured covariance matrices. A closed-form solution for the estimation Hermitian Toeplitz matrices is obtained which makes the proposed method computationally much simpler than most existing Hermitian Toeplitz matrix estimation algorithms. Additionally, it was also shown that using the technique in such array processing algorithms as MUSIC [64] and ESPRIT [63] makes them achieve the Crambr-Rao Bound (CRB) for angle estimation, i.e., the best performance for any unbiased methods. CHAPTER 3 MAFI APPROACH TO SPECTRAL ESTIMATION Filterbank approaches decompose the observations {y(n)} 1 of a stationary signal y(t) as [51] [47] [73] [72] y(n) = a(w)ewn +e,(n), n = 0,1,...,N 1; w [0,2r), (3.1) where a(w) denotes the complex amplitude of the sinusoidal signal with frequency w and e,(n) denotes the noise (or residual) term at frequency w, assumed to be zero-mean. The problem of interest is to estimate a(w) for any given w. Briefly stated, most filterbank spectral approaches address the aforementioned problem by following two main steps: (a) pass the data {y(n)} through a bandpass filter with varying center frequency w; and (b) obtain the estimates, &(w), for w E [0, 27r), of the complex amplitude from the filtered data. The bandpass filter used is usually an M-tap FIR filter with its coefficient vector given by h = hi h2 ... (3.2) where (.)T denotes the transpose. (The choice of M is discussed in Section 3.5.) Observe that the notation emphasizes the dependence of the vector in (3.2) on the center frequency w. Although rules for choosing he vary, a rather general one for the choice of a matched filter is discussed in Section 3.2. 3.1 Forward and Backward Data Vectors Let y(1) y) ( ) y( M-) 1, (3.3) 16 17 be the overlapping vectors constructed from the data {y(n)}, where L = N-M+1. In what follows y(1) is referred to as the forward data vector. Let e,(1), 1 = 0, 1,..., L-1, be formed from {e,(n)} in the same manner as y(l) are from {y(n)}. Then the forward vectors can be written as y(1) = [a(w)aM(w)]ewl + (1), (3.4) where aM(w) is the steering vector and is given by aM(w) = 1 e ... ej(M-1) (3.5) Likewise, the backward data vectors are constructed as (1) y*(N-1-1) y*(N-1-2) ... y*(N - M) S= 01,. L 1, (3.6) where (-)* denotes the complex conjugate. Let e,(1), 1 = 0, 1,..., L 1, be formed from {e,(n)} the same way as (1l) from {y(n)}. Then the backward vector can be written as ?r(1) = [d(w)aM(w)]ejwil e,(1), (3.7) where d(w) = a*(w)e-(1)W. (3.8) It is straightforward to verify that the forward and backward vectors are related by the following complex conjugate symmetry property: k(1) = Jy*(L 1 1), (3.9) where J denotes the exchange matrix whose anti-diagonal elements are ones and all the others are zero. Suppose that the initial phase of the sinusoidal signal in (3.1) is a random variable uniformly distributed over the interval [0, 21r) and independent of the noise 18 term. By making use of this assumption as well as (3.9), the covariance matrix of y(1) or, equivalently, of f(1), is given by R ~ {y(l)yH(1)} = E {(l)H(1)} = 2a()a)HM(w) + Q(w), (3.10) where (.)H denotes the conjugate transpose and Q(w) is the noise covariance matrix and is given by Q(w) E H()} = E {()H()} (3.11) Note that both R and Q are Hermitian Toeplitz matrices. The Forward-Only (FO) filterbank approaches use only the forward data vector to estimate R, which is the forward sample covariance matrix: 1 L-1 RFO = Y'(1)yH(1). (3.12) 1=0 The Forward-Backward (FB) filterbank approaches use the average of the forward and backward sample covariance matrices to obtain the estimate of R: RFB = 1(RFO + iBO), (3.13) where BO denotes the backward sample covariance matrix: 1 L-1 RBO = Jl(H(1)H = JftoJ. (3.14) 1=0 The RFB in (3.13) is Hermitian but no longer Toeplitz. By using (3.9), one can show that RFB is a persymmetric matrix [30], i.e., IFB = JFBJ. (3.15) Since R is persymmetric, one would expect that RFB is a better estimate of R than the non-persymmetric RFO. 19 3.2 MAFI Filters By definition, the matched filter is designed such that the corresponding signal-to-noise (SNR) ratio in the filter output is maximized; that is, hw = arg max h (w) (3.16) he hHQ(w)h, The solution is obtained by making use of the Cauchy-Schwartz inequality (see, e.g., [75]): Q-' (w)a(w) he = (3.17) where Q(w) is assumed to be invertible. It is readily checked that the solution in (3.17) satisfies h HaM(w) = 1, (3.18) which implies that the filter given in (3.17) passes the frequency w undistorted. By making use of this observation and of (3.4) and (3.7), we have hH( (w)' + hH (), I = 0, 1, ..., L 1, (3.19) and hH~) =e-j(N-1)W (w) +hHe,(1), 1= 0,1,..., L 1. (3.20) The least squares (LS) estimate of a(w) obtained by using only (3.19), i.e., the forward data vectors, is given by FO(W) = hHg(w), (3.21) whereas the least squares (LS) estimate of a(w) obtained by using (3.19) and (3.20), that is, both the forward and backward data vectors, is given by <^FB(W) = 2 [hHg(w) + e-j(N-)wgH(w)h] (3.22) where g(w) and g(w) are, respectively, the normalized Fourier transforms of the forward and backward data vectors: 1 g(w) = Ey~(1)e, (3.23) 1=0 20 1 L-1 g(w) = y(1)e-j"W. (3.24) 1=0 Since Q(w) is Toeplitz and, therefore, persymmetric, one can show that the h, in (3.17) satisfies [52] Jh* = hee-j(M-1)w. (3.25) Consequently, it follows after some calculation that (3.22) is equivalent to &FB(W) = hH (w). (3.26) Hence, due to the persymmetry of Q(w), the FB estimate of a(w) has the same form as the FO estimate of a(w) (see (3.21) (3.26)) However, the filter vector h, obtained with the FO approach is in general different from that corresponding to the FB approach [52]. Although neither Capon nor APES was derived in the MAFI framework (for original derivations of these methods, we refer to [16] [75] [52]), in what follows we show that two natural estimators of Q(w) in (3.17) lead to the Capon and APES filters, respectively. More interestingly, we also show that even though a third natural estimator of Q(w) gives a new filter which is different from the former two, the spectral estimator corresponding to the new filter turns out to be equivalent to APES as well. 3.2.1 Capon Filter By (3.10), one natural choice is to estimate Q(w) as QCapon(W) I a(w)12 aM(w)aH(w), (3.27) where &(w) denotes an estimate of a(w) and R denotes either RFO or RFB, which, in turn, leads to FOC or FBC, respectively. (For notational simplicity, in what follows the forward-only Capon, APES, and MAFI spectral estimators will be referred to as FOC, FOA, and FOM, respectively. Likewise, FBC, FBA, and FBM represent the corresponding forward-backward counterpart estimators.) By making use of the 21 matrix inversion lemma [30], one can see that the second term in (3.27) has no influence on the he in (3.17). Hence, when (3.27) is substituted into (3.17), the matched filter reduces to the Capon filter [16] [44]: hCapon -'aM (w)(3.28) a (w) aM ) (3.28) Observe that Q(w) is persymmetric for either FOC or FBC. By substituting (3.28) into (3.21) or (3.26), we obtain the Capon estimate of ca(w): am (w) 1(w) &Capon(w) = aH(w)RlaM(w ) (3.29) 3.2.2 APES Filter Ignoring the fact that aM(w) is known, we obtain the LS estimate of the vector a(w)aM(w) in (3.4) as [a(w)aM(w)] = g(w). (3.30) Inserting (3.30) into (3.27) along with RFO substituted yields the FOA estimate of Q(w): QFOA(W) = IFO g(w)gH(w). (3.31) A persymmetric estimate of Q(w) can be obtained by using both the forward and backward data vectors: QFBA(w) = + JQFOA(w)J = RFB G(w)GH(w), (3.32) where G(w) = g(w) g() (3.33) and we have used the fact that Jg*(w) = ej(L-1)g(w). Hence by (3.17), we obtain the the APES filter [52]: hAPES = Q-(w)aM(w) h' (w)Q (w)a(w)(3.34) aM ()-1 )RM 22 where Q(w) denotes either QFOA or QFBA which corresponds to FOA or FBA. Consequently, the APES estimate of a(w) is given by (see (3.22) and (3.26)) &APES(w) = (3.35) 3.2.3 Another Matched Filter Equations (3.4) and (3.7) suggest another way to estimate the FO and FB estimate of Q(w) (in what follows we sometimes omit the dependence on w for notational convenience): QFOM( ) 1 L-1 L= [y(l) &(w)aM(w)e"w] [Y(l) &(w)aM(w)ejw ]H 1=0 = RFO a -aaMgH + &aMaHM, (3.36) nFO QFBM (w) L-1 2L E (Y() &(w)a(w)e ) (() &(w)a(w)ew)H 1=0 + (() a(w)aM(w)e3w) ((1) (W)aM(w)eiw)H] = IFB [& + g a H HM1 M[& ~1H g+12 aMaHM, (3.37) QFB where &(w) and &(w) = &*(w)e-j(N-1) denote some estimates of a(w) and &(w), respectively. By using the matrix inversion lemma (twice), one can see that the last and the third terms of (3.36) or (3.37) can be dropped without affecting the matched filter vector. Then, by using the matrix inversion lemma once again, we have F aM = (RIFO &*aH)-1 = a R-1am &*(aH O1 am + &*(aH Oa)R O = FO M F0- 41 H- I UF-I (3.38) 1 &*aHm FO 23 and g + a a ] aM = -F am= + FB [g + ~RR (&*g + HaM FB S1FOM ,FOa 1 a- a HM M Fa + &*ag aM &*aHR aH Fa lft-am Hlf1 a A-- am + 1 H ,-1 amft-1 * hFBM- 2a FB + 2aMFBa FB S+ aFB [&*g + (3.39) which gives the following expressions for the matched filter vectors: hFFOM OMFOA() (3.42)aM Hm(l) = Aa (w). (3.43)1 f-'am + a aM FoaMRfo-lg *a aM1FogRFOaM(34 aHROFOaM (3.40) and hFBM FBaM aM 'FBaM Fa 2 M FaMFB [&*g +1 __ 2* ] l+FhaM amRFB aM (3.41) The previous MAFI filters are in general different from both the Capon and APES filters, since neither the Capon nor the APES filters depend on an estimate of a(w) which the new MAFI filters need to know. In spite of this fact, in Appendix A, we prove that, for a certain natural choice of &(w) in (3.36) and (3.37), the following equalities hold true: &FOM(W) = 6FOA(W), (3.42) &FBM(W) = &FBA(w). (3.43) 24 3.3 Analyses of MAFI Approaches 3.3.1 Computational Complexity Let fF1/2 and ft /2 denote the Hermitian square roots of the positive definite matrices R f) and i1F, respectively. Define FO FB--D i (W) = FO/2 aM(w), (3.44) 2() = tF2g(), (3.45) Then FOC and FOA can be expressed as relatively simple functions of ,l (w) and pz2(w) (see (3.29) and (3.35)): Foc() =C1 \( ) 2( ) 11 1( ) 112 (3.46) and HFOA)(w) = (3.47) FOA(W) 1(w)~2 (l1 (W)12 112 H2 (w)2()12 Applying the matrix inversion lemma to (3.32) yields BA(W) R G(w) GH ) G(w) I-GH (w)t (3.48) where I is the 2 x 2 identity matrix. Next define Vl(W) = FB/2aM(w), (3.49) V2(W) B= F/2g(w), (3.50) v3(W) = /2g(w). (3.51) Then the FBC and FBA spectral estimators can be expressed as (see (3.29), (3.35), and (3.48)): &FBC(W) =2 IvHw)V2W) (3.52) 25 and x i. [~. (u ~ L) l V 2(W ) 12 1'(W)V2(W) 1 H'W () H'(w)V(W)] ^FBA(W) = Il(W)i12 I 2H2 1 2 I 1 W)2() H(()w3(-)] 1 (3.53) where 1 II2() 112 2 (W) 3(W) (w) = 2 Il(W)V2(W) HV3()I I. (3.54) Hence computationally APES, especially FOA, is only slightly more involved than Capon. (Also see Section 3.5 for the simulation results.) More specifically, the amount of computations required by Capon or APES is dominated by calculating fl-1/2 and the matrix-vector products in (3.44)-(3.45) or (3.49)-(3.51). We mention that conventional Capon or APES implementation makes use of ((3.46), (3.47)), ((3.52), or (3.53)), which requires calculating the matrix-vector products in (3.44)(3.45) or (3.49)-(3.51) for each w of interest, thereby becoming computationally more and more intensive as the number of frequency samples increases. This is especially so in 2-D applications such as when forming SAR images. It is thus of great interest if other efficient implementation schemes for the MAFI approaches can be found. 3.3.2 Statistical Performance All estimators under study, i.e., FOC, FOA, FBC, and FBA, can be shown to have the same asymptotic variance under the following condition: C: The signal y(n) can be written as in (3.1), where e,(n) is a zero-mean stationary random process with finite spectral density at w: e(W) < 00. (3.55) In more exact terms, the following result holds true. 26 Theorem 3.3.1 Under Condition C and the additional assumption that e,(n) is circularly symmetrically distributed, the estimation errors in the Capon and APES spectral estimators are asymptotically circularly symmetrically distributed with zeromean and the following common variance: lim LE {I&(w) a(w)|2} = e(w). (3.56) L--oo Proof: See Appendix B. The need to enforce Condition C limits, to some extent, the importance of the previous result. Indeed the assumption made in C is satisfied if (and essentially only if) the signal y(n) has a mixed spectrum and w is the location of a spectral line. The result of Theorem 1 is relevant to the spectral analysis of a target with dominant point scatterers in the presence of distributed clutter (see [52] and the references therein). In some other applications, however, the main interest is in the continuous component of the spectrum. For example, Condition C does not hold exactly for a target with distributed scatterers since the signature spectrum is continuous at w. That the previous result is of a somewhat limited interest is also due to its asymptotic character. Indeed, in applications with medium or small-sized data samples, the spectral estimators under study have been found to behave quite differently in contradiction with what is predicted by the (asymptotic) result of Theorem 1 (see the numerical examples in Section 5). The finite-sample analysis of the spectral estimators under discussion would consequently be of considerable interest. However, a complete analysis, if possible, appears to be rather difficult at best. A partial one, by making use of a higher-order Taylor expansion technique, is nevertheless feasible. The result is as follows. Theorem 3.3.2 To within a second-order approximation and under the mild assumption that the third-order moments of e,(n) and e,(n) are zero, the Capon and 27 APES estimators are related by LE {&FBC(W) a(w)} LE {&FOC(W) aM(w)} (< 0w (3.57) and LE {&FBA(W) a(w)} = LE {&FOA(W) a(w)} = 0, (3.58) for sufficiently large values of L. Proof: See Appendix C. We believe that (3.57) and (3.58) provide a theoretical motivation for preferring APES to Capon in most spectral estimation exercises. Moreover, Theorem 2 also suggests that FBC should be preferred over FOC. While both FOA and FBA are similarly unbiased (within a second-order approximation), the latter is usually observed with slightly better resolution and sidelobe properties [52] at the cost of slightly more computations. 3.4 2-D Extensions We briefly describe the 2-D extensions of the MAFI spectral estimators. We first decompose the observations {y(ni, n2)} as y(ni, n2) = 1(W1, W2)ej(wlnl+w2n2) + e,(n, 12), (359) (3.59) n =, 1,...,N-1; n2 =0, 1,...,N2 1; W1, W2 E [0,27r), where a(wl, w2) denotes the complex amplitude of a 2-D sinusoidal signal with frequency (wl, w2) and e,,,, (nl, n2) denotes the noise (or residual) term at frequency (w, w2), assumed to be zero-mean. Next, in a manner similar to the 1-D case, we form the M1 x M2 forward and backward data matrices: Y(I1, 12) = y(nl,n2), n= 1, 1 + M -+M- 1; n2 = 12,...,12 + M2 Y(I1, 12) = y*(nl,n2), n1 = 1 ,..., N 11 M1; n2 = N2 12 1,..., N2- 12- M2} , 11 =0,1,..., L 1; 12=0,1,...,L2-1, (3.60) 28 where L =N1 M1 +1 and L2 =N2 -M2 + 1. Let Y(11, 12) = vec[Y (1,12)], (3.61) :(l, 12) = vec[rY(1,/2)], (3.62) where vec[-] denotes the operation of stacking the columns of a matrix on top of each other. Let aM1,M2 (W1, W2) = aM2 (W2) aM 1(1), (3.63) where 0 denotes the Kronecker matrix product, and aMk k) = 1 eW ... ej(Mk-1)k k = 1,2. (3.64) Then Y(11,12) and S(l1, 12) can be written as Y(ll, 12) = [0(wl,,W2)aM1,M2 (w 2)]ej(w1li+212) + ek ,2 (l, 12), (3.65) r(11, 12) = [&(Wl W2)aM,M2(w1, W2)l]e(wl l1+212) + 1,W2(11, 12), (3.66) where &(w1, w2) = *(w,2)ei-j(N-1)we-j(N2-l) 2, (3.67) and 4,, (11, 12) and e~,,,2 (i, 12) are, respectively, formed from {e,,,, (ni, n2) } in the same ways as Y(11, 12) and y(l1, 12) are made from {y(nj, n2)}. Suppose that the initial phase of the sinusoidal signal of (3.59) is a random variable uniformly distributed over the interval [0, 27r) and independent of the noise term. Then the covariance matrix of y(1~, 12) or, equivalently, of k(l1, 12) is given by R = la(wl, W2)2aM1,M2(W1,W2)aM1,M2 (W1, W2) + Q(W1,W2), (3.68) where Q(wl,W2) is the covariance matrix of e,1 (11, 12) or e61,, (11, /2). By making use of the fact that r(l1,/2) = Jy*(L1 11 1, L2 12 1), (3.69) 29 one can see that R is persymmetric. Similarly, Q is also persymmetric. The forward-backward sample covariance matrix takes the form: iFB = I(RFO + IRBO), (3.70) where FO and RBO denote the sample covariance matrices of {y(l1, 12)} and {((11, 12)}, respectively, given by SL1-1 L2-1 RFO = (11,12)H1, 2), (3.71) 11=0 12=0 L1-1 L2-1 RBO = (1112)yH(1 12). (3.72) 11=0 12=0 By making use of (3.69), one can see that RFB is also persymmetric. Let H,,,,2 denote the impulse response of an M1 x M2 2-D FIR filter, and let hl,.2 = vec[H,,,,,2]. (3.73) Like in the I-D case, the impulse response of the matched filter is given by Q-1 (w1, W2)aM1,M2 (w, w2) = a H,M2 (1, W2 Q-1(W, W2)aM1,M2 (W1, W2) 7 Note that hH,2 aM1,M2(W1,W2) = 1. (3.75) The LS estimates of a(w1, w2) obtained by using only the forward data vectors and by using both the forward and backward data vectors are given by (similarly to (3.21) and (3.22) in the 1-D case) FO(W1, W2) = h,'g(w), (3.76) and FB 1, W2) [h1,W2g(wl, 2) + e-j(N-l1)w ej(N2-l)w2gH(1, W2)hw,,w2] (3.77) 30 where SL-1 L2-1 E(w1,w2) LL2 Z Y(, 12)e-j(w'' w2'), (3.78) 11=0 12=0 L1-1 L2-1 (w,0w2) LjL2 1 2 Y(ll, 12)e-j(ll+w1 212). (3.79) 1i=0 12=0 Since Q is persymmetric, (3.77) can be written as &FB (1, W2) = h~ 1, 2). (3.80) The Capon method estimates the noise covariance matrix as (Capon (W1, W2) = i- IL(1,2) I2aM,M2 (w1,2)aH,M2( W1, 2), (3.81) where &(w1, w2) denotes some estimate of a(W1, w2), and R denotes either RiFO or IRFB, which correspond to 2-D FOC or FBC. Thus, the Capon estimate of a(w1, w2) is obtained as &Capon(w1, 2) = aMI'M2 (W1, 2) 1- 2 ((3.82) aMl ,M2 (W1, w2)i-laM1,M2 (W1, W2) The FO and FB APES estimates of Q(w1, w2) take the form: QFOA (1, W2) rFO (w1,W2)H(W1, W1), (3.83) and QFBA (w1,2) = FB G(w1, 2)GH (W, W1), (3.84) where G(w,W2) = 2g(W,L02) 9(l, 12) (3.85) Hence we obtain &APES (W1, w2) as aHM ( 1, l) -1J( 1, L)l( 1 2) &APES (W1, W2) =- M21, 2(3.86) a .1 (W, 12)Q -1(W1, 2)aM1, ((,r 1 2) where ((i,W2) denotes either QFOA(Wi, 2) Or Q(FBA 2) - 31 Based on the 2-D extensions described above, it is not difficult to see that all the results of the previous section also hold true for the 2-D Capon and APES estimators. Indeed, the proofs for the 2-D estimators follow a similar pattern to those for the 1-D case shown in Appendixes B and C. 3.5 Numerical Examples In the following, we study the Capon and APES complex amplitude estimates in a number of cases of interest. For both the 1-D and 2-D examples given below, we compare the performance of the forward-only Capon and APES as well as the forward-backward Capon and APES, which are, for simplicity, referred to as FCapon, FAPES, FBCapon and FBAPES, respectively. 3.5.1 1-D Complex Spectral Estimation The 1-D data used in the examples consists of a sum of 15 complex sinusoids, with the real and imaginary parts shown in Figures 3.1(a) and 3.1(b), respectively, corrupted by a zero-mean complex white Gaussian noise. The data length is chosen as N = 64. In what follows we are interested in the bias and variance properties of the estimators under study. The bias and variance results shown below correspond to the frequency of the first sinusoid and they are obtained from 100 independent realizations. We begin by studying the performance of the estimators as the signal-to-noise ratio (SNR) varies. The SNR for the kth sinusoid is defined as SNR = 10log1 a (dB), (3.87) 0 e(Wk) where COk is the complex amplitude of the kth sinusoid and Pe(wk) is the spectral density of the additive noise at frequency wk. The filter length is chosen as M = 15. The real and imaginary parts of the bias are shown in Figures 3.2(a) and 3.2(b), respectively, as a function of SNR1. As seen from these figures, FOA and FBA are 32 almost unbiased, while FOC and FBC are biased downward. In addition, we notice that the bias for FOC is approximately twice that of FBC. All these observations are consistent with the prediction of the theory. The variances of the real and imaginary parts of the amplitude estimates are shown in Figures 3.2(c) and 3.2(d), respectively. It appears that all of the estimators display similar variances. However, as shown in the next example, the variance of Capon becomes notably larger than that of APES as M increases. Next we study the effect of the filter length, M, on the estimators. The SNR1 is fixed at 20 dB. As M varies, the real and imaginary parts of the bias are shown in Figures 3.3(a) and 3.3(b), respectively. From these figures, one can see that both FOA and FBA are unbiased for all practical filter lengths, whereas the bias of Capon grows significantly with increasing M. (A practical filter length means that M should not be too small [52]. In fact, all filterbank methods reduce to the Fourier transform approach when M = 1, and only when M is sufficiently large, the filterbank approach shows noticeable improvement over the Fourier method [52].) All estimators seem to perform similarly for M up to a fourth of the data length, with Capon being slightly biased downward. As the filter length increases further, the performance of Capon degrades rapidly, while that of APES remains unaffected. This observation is strengthened by the variance results shown for the real and imaginary parts of the amplitude estimates in Figures 3.3(c) and 3.3(d), respectively. It is known that, as M increases, all of the estimators under study achieve better spectral resolution and that the best resolution is obtained at M = N/2 [52]. This fact, along with the statistical results shown in the previous examples, indicates that the choice of M for Capon should be made by a tradeoff between resolution and statistical stability. Usually we choose N/4 < M < N/2. While the choice of M for Capon is difficult to make, it is easy to see that APES achieves the best performance at M = N/2, since with this choice, APES achieves the highest possible 33 1-D Complex Sinusoids 1-D Complex Sinusoids 0.9 0.9 0.8 0.8 r V 0.7 0.7 0.6- 0.6 I .5I I 0.5 (a) (b) = 0.4 0.4 0.1 -D complex amplitude of the sum of 15 sinusoids 0.1used in the 0 0.2 0.4 0.6 0.8 1 0 0.2 OA 0.6 0.8 1 Frequency Hz Frequency Hz (a) (b) Figure 3.1: The 1-D complex amplitude of the sum of 15 sinusoids used in the simulations. (a) Real part; (b) Imaginary part. resolution as well as the best statistical properties in terms of bias and variance. The previous examples also show that FOA and FBA perform similarly in terms of bias and variance properties for the frequency of interest. To compare the computational complexities of the estimators under study, we count the flops required by each of them for the case where N = 64, M = 24, and the complex spectra are evaluated at 256 equally spaced points. The flops required by FOC and FBC are approximately the same, whereas the flops needed by FOA and FBA are, respectively, 1.08 and 1.41 times of that by the Capon estimators. 3.5.2 2-D Complex Spectral Estimation As was mentioned in Section 3.4, the 2-D Capon and APES estimators behave rather similarly to their I-D counterparts. Since the problems encountered in applications such as synthetic aperture radar imaging are concerned with 2-D complex spectral estimation, we include a couple of 2-D numerical examples here. The data employed consists of three 2-D sinusoids corrupted by a 2-D zero-mean complex white Gaussian noise, with N1 = N2 = 32. The sinusoids are located in the frequency domain at (0.2, 0.2), (0.25, 0.25) and (0.4, 0.7) and their amplitudes are ej,/4, ej1/4 34 x10-a 1-D Complex Amplitude Estimation 1-D Complex Amplitude Estimation 5 I 0.015 oo o. 9 I-- FOC 94". ': :' 0"0, o 0.01 0 a 0.01 - 0 .FBC -0 00 '0' 0.0 0 S-0.01 -15- FBC -0.01 5 10 15 20 25 30 5 10 15 20 25 30 SNR dB SNR dB (a) (b) x 10-3 1-D Complex Amplitude Estimation 10- 1-D Complex Amplitude Estimation FOC FOC --- FC 1. --- FC + FOA + FOA 2 FBA 1.4 o o FBA 0. 0. 1.2 1.5 E 1 a_0.8 0.6 0.5 0.4 0.2 ( C 5 10 15 20 25 30 5 10 15 20 25 30 SNR dB SNR dB (c) (d) Figure 3.2: Empirical bias and variance of the 1-D MAFI estimators as SNR1 varies when N = 64 and M = 15. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. 35 1-D Complex Amplitude Estimation 1-D Complex Amplitude Estimation 0.1 0.3 0.2.---0.1 n0 ... o ... *o -0.1 0 -0.4 FOC - FBC -0.4-0.5 o FBA -0.5I I I I I I -0. 5 10 15 20 25 30 35 0.60 5 10 15 20 25 30 35 Filter Length Filter Length (a) (b) 1-D Complex Amplitude Estimation 1-D Complex Amplitude Estimation 0.012 0.012 FOC FOC 0.01 --- FBC 0.01 - FBC * FOA + FOA o o FBA I o FBA 0.008 - 0.008 i 0.006 o0.006 >I > 0.002 0.002 0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35 Filter Length Filter Length (c) (d) Figure 3.3: Empirical bias and variance of the 1-D MAFI estimators as the filter length, M, varies when N = 64 and SNR1 = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. 36 and 0.7ej,/4, respectively. The bias and variance for the amplitude estimate of the first 2-D sinusoid are obtained from 100 independent realizations. The SNR for the kth 2-D sinusoid is similarly defined as in (3.87). The bias and variance of the four estimators under study versus SNR1 are shown in Figures 3.4(a) to 3.4(d), respectively, where M1 = M2 = 8. Figures 3.5(a) to 3.5(d) show the statistical results as the 2-D FIR filter length varies, where SNR1 is fixed at 20 dB. We assume in Figures 3.5(a) to 3.5(d) that M, = M2. As seen from these plots, the performance of the 2-D MAFI estimators indeed resembles that of their 1-D counterparts and, therefore, we refer the readers to the 1-D examples for comments. 3.6 Summary This chapter discusses using the MAFI approach for complex spectral estimation. The Capon and APES estimators are shown to be members of the MAFI class. By using a higher-order expansion technique, it is proved that to within a secondorder approximation Capon is biased (downward) while APES is unbiased, and that the bias of the forward-backward Capon is one half that of the forward-only Capon. It is also shown that the above conclusions carry over to the 2-D MAFI estimators as well. Since computationally APES is only slightly more involved than Capon, the preference of APES to Capon in practical applications follows logically because of the better statistical properties associated with the former. 37 2-D Complex Amplitude Estimation 2-D Complex Amplitude Estimation 0.005 0.005 0.* . 0 : ........ O. "1 : ". o. .,.o .. 1 is * SNR dB SNR dB SComplex Amplitude Estimation Complex Amplitude Estimation -- FBC - FBC -0. FOA --- FOA 2 a o FBA a FBA 0.025 0.025 0.5- -0.4 5 0 5 10 15 -5 0 5 10 15 SNR dB SNR dB (a) (b) 2.Figure 3.4: E2-D Complex irical bias and variance of the 2-D MAFIeAmplitude Estimat honors as the SNR varies when NFC N2 = 32 and M 8. (a) Real part of the bias; (b) (d) Variance of the imaginary part of the estimated amplitude.FBA 5 0 5 10 15 -5 0 5 10 15 SNR-dB SNR-dB (c) (d) Figure 3.4: Empirical bias and variance of the 2-D MAFI estimators as the SNR1 varies when NI = N2 = 32 and Mt = M2 = 8. (a) Real part of the bias; (b) 38 2-D Complex Amplitude Estimation 2-D Complex Amplitude Estimation 0.1 iI I 0.1 S-0.1 -0.1 -0.3- -0.3 '-0.4 FOC -0.4 F-FOC A -0.54 FBA -0.5-0.c I I I 0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16 Filter Size Filter Size (a) (b) x10' 2-D Complex Amplitude Estimation 10- 2-D Complex Amplitude Estimation FOC FOC - FBC - FBC FOA + FOA 6 OFBA 7 0 OFBA U 4 I 53- 1 1 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16 Filter Size Filter Size (c) (d) Figure 3.5: Empirical bias and variance of the 2-D MAFI estimators as the filter length, M = M1 = M2, varies when N1 = N2 = 32, and SNR1 = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. CHAPTER 4 EFFICIENT IMPLEMENTATION OF CAPON AND APES 4.1 Introduction In the previous chapter, we rederived the Capon and APES spectral estimators using the MAFI approach. In the MAFI framework, a number of statistical properties of Capon and APES were obtained. However, we did not address very carefully the implementation of the Capon and APES estimators. Equations (3.46), (3.47), (3.52), and (3.53) give the intuitive ways of implementing Capon and APES. However, such intuitive implementations are computationally expensive, especially for 2-D spectral estimation from 2-D data sequences. In this chapter, we study how to implement Capon and APES efficiently. For simplicity's sake, we only consider 2-D spectral estimation since 1-D spectral estimation is a special case of the former; we also only consider forward-backward Capon and APES since they are more often used than their forward-only counterparts. 4.2 Efficient Implementation of APES First we rewrite the 2-D forward-backward APES estimator as aM,M2 (W1, (2)1-l(W1, W2) (U1, W2) &APES(w, w) = a (4.1) L1L2HM1,M2 (W1, W2) -1 (W1, w2)aM1,M2 (W1, W2) where (W, K [(w) ftw, )+ ( ) [W, C(WA] (4.2) with fR denoting the forward-backward sample covariance matrix given by (3.70), Li-1 L2-1 (W1,0)2)= (ll1,2)e-(1I1+w212), (4.3) 11=0 12=0 39 40 and L1-1 L2-1 I(wl,w2)= 7 1 2)e-j(wil+w212). (4.4) [1=0 12=0 Let Z = [ y(0,0)... y(L- 1,0)... (0, L2 -1)- (Li-1, L2 1) (4.5) and Z = JZ*J, where J denotes the exchange matrix. We can then rewrite (4.3) and (4.4) as (W1, W2) = Za*L,L2 W2), (4.6) and I(w1,w2) = ZaL1,L2 (1, W2) (4.7) By applying the matrix inversion lemma, we obtain (1, 2) -1(w1, 2) + Q-(W1, 1, 2) (1, W2)Q(W1, W2) (4.8) L1L2 i (W1, W2)Q-1 (W, 12)I(WI, W2) where R (, w2)IP H()Q-1(wl, 2) = I + H (4.9) LL w (W1, 2)R-1R(w1, W2) (For notational convenience, we sometimes drop the dependence on w1 and w2 below.) Hence, aHM,,M2 (1, 1)Q-1 (W1, 02) (1, W2) H H-1 aMt,M2Q H+ I jL H --1SL MM2Q (4.10) L1L2 HQand aM2 (w1, W2) -1aM,M2 (W1, W2) H --I + aM,M2Q aM,M2 a1,M2 + (4.11) L1L2 1 Q-'V) 41 It follows that (4.1) can be rewritten as (L1L2 H- aM,M2Q- aM1,M2+ aM1,M2Q2 Since R is Hermitian and positive definite, we can obtain an upper triangular matrix C by Cholesky factorization such that 1-1 = ( -1)H [30]. Let bT(w1, w2) a,M2 2)C-, (4.13) d(wi,w2) = DaL,L2(Wl, 2), (4.14) and e(wl, w2) = Ea*L,L2 (wl, w2), (4.15) where D = (-)H, (4.16) and E = (C)HZ. (4.17) We have [53] aMH,M2 aM1,M2 I P HH M -1 H -1 =1 M1,2 ~ +Rt-'L bbTw2) b(ww2)e(, W2)H (W1, w2)d(w, w2) = bb(Wl, W2)d(W),2)2)w2), (4.18) L1L2 le(wl,w2) 112 H Q-1p = H f-1 + L1L2 1Hi-l' = d(, 2) 112 + W 2)e(w 2) 2 (4.19) L1L2 jle(wl,W2) 112 42 and aM1,M2Q-1 aM,M2 H -1 H, -1M2 H -1MM2 aM1,M2 m, M1 ,M2 J ,2 LIL2 R a = Ilb(w,,w2)12 + bT(wj,W2)e(wj,W2)(0 = 2b(WWe(ww2)22 (4.20) L1L2 Ie(w,w2)I12 Next we observe that b(wl,w2), d(wi,w2), and e(wi,w2) can be calculated via 2-D FFT. Specifically, we partition C-1, D and E as follows: -1= vec{B1} ... vec{BMM2} (4.21) D= vec{Di} ... vec{DM 2} (4.22) and E= vec{Ei} ... vec{EM1M2} (4.23) where Bk C CM1XM2, Dk E CLlxL2 and Ek E CLjxL2. Then (again, we drop the dependence on wl and w2) b= aM1 ,M2 (a2 H ai )vec{B} ... (a12 aH)vec{B jm2} = aHMBMa2 ... aH1BM Ma 2 a* (4.24) Likewise, we have d Da1 = D aH 1Djia ... aH DMM2a* (4.25) and e = Ea,,,2 = aH Ea2 ... aEMM2a2 .T (4.26) Note that aH Bka* represents the 2-D discrete Fourier transform of Bk at (w, w2). Hence APES can be efficiently implemented by first calculating b(w1,w2), d(wl, w2) 43 and e(wi, w2) via 2-D FFT, then using Equations (4.18), (4.19), and (4.20) to determine aH,,M2Q-1, ~HQ-1 and aH,M2Q- aM1,M2 respectively, and finally using (4.12) to obtain &APES(Wl,W2). The structure of (3.53) is similar to that of (4.12). However, the amount of computation required by the former is much larger than that by the latter. The reason is that, even though 4(w1, w2) and P(l,W 2) can be obtained by 2-D FFT, for each (wl, 2) pair, we have to compute the additional matrix-vector products t-l/21(wl, w2) and W-1/2 M1,MM (W1, W2) (recall that E-i/2 E CM1M2 M1M2, 2(W1, 2) and 'I(w1,W2) E CM1M2x1) to obtain A2(w,2 W2) and pa (w1,w2). On the other hand, by computing D and E first (which are computed only once), we bypass calculating such matrix-vector products and save a large amount of computation. The larger the number of samples in the 2-D frequency domain, the more the amount of computation we will save. These discussions also apply to the implementation of Capon. 4.3 Extension to Capon Note that APES becomes Capon when Q(wi, w2) is replaced by fR. Hence the efficient implementation of Capon can readily be achieved by modifying (4.12) as follows: bT (w, w2)d(wl, 0w2) Capon(w1, 2) (w, w2) (4.27) LL2 IIb(wl, w2)112 More specifically, the efficient implementation of Capon is by using (4.13) and (4.14) to calculate b(wl, 02) and d(wl, w2), respectively, and then using them in (4.27). Since the amount of computation required to calculate b(w, C) in (4.13), d(wi, w2) in (4.14), or e(wi, w2) in (4.15) is approximately the same and calculating &APES(W1, 2) by (4.12) and &Capon(W1, W2) by (4.27) are much less involved than obtaining b(w1, w2), d(wl, w2), and e(wi, 02), the total amount of computation required by APES is about 1.5 times of that required by Capon, as verified by the numerical and experimental examples in Section 4.4. 44 For similar reasons as for APES, the intuitive implementation of Capon given in (3.52) is computationally more involved than the efficient implementation of Capon proposed above. 4.4 Numerical and Experimental Examples We present numerical and experimental examples comparing the performances of APES and Capon with the FFT methods [52] for SAR imaging. In the following examples, we choose M = N/2 and M = N/2 for both Capon and APES. For the windowed FFT method, we use the Kaiser window with parameter 4. We first consider SAR imaging of a simulated MIG-25 airplane. The 32 x 32 data matrix was provided by the Naval Research Laboratory. The 128 x 128 SAR image obtained by using 2-D FFT, 2-D windowed FFT, 2-D Capon, and 2-D APES are shown in Figures 1.1(a) to 1.1(d), respectively. We note that Capon and APES outperform the FFT methods. The number of flops required by our efficient ways of implementing Capon and APES are about 950 and 1500 times of those required by the FFT methods, while those required by the intuitive ways of implementing Capon and APES are about 22800 and 30000 times, respectively, of those required by the FFT methods. That is, the number of flops required by the intuitive ways of implementing Capon and APES are, respectively, about 24 and 20 times of those required by our efficient ways of implementing them. If we increase the size of the image to 256 x 256 and 512 x 512, respectively, the ratios of the needed flops between the intuitive ways and our new ways for implementations are 36 & 40 for Capon, and 31 & 34 for APES, respectively. We now consider an example of SAR imaging with experimental data. The data matrix is 64 x 64 and is obtained from the experimental data collected by one of the two apertures of the ERIM's (Environmental Research Institute of Michigan's) DCS IFSAR (interferometric SAR). The 256 x 256 SAR image obtained by using 2-D 45 FFT, 2-D windowed FFT, 2-D Capon, and 2-D APES are shown in Figures 4.1(a) to 4.1(d), respectively. Again, Capon and APES outperform the FFT methods. The number of flops required by the intuitive ways of implementing Capon and APES are, respectively, about 38 and 32 times of those required by our efficient ways of implementing them. If we increase the size of the image to 512 x 512, the ratios of the needed MATLAB flops between the intuitive ways and our new ways of implementing Capon and APES are 86 and 73, respectively. 4.5 Summary This chapter addresses the implementation of the Capon and APES spectral estimators. The amount of computation required by APES is shown to be about 1.5 times that required by Capon. By using a technique proposed in this chapter, the computational complexities of Capon and APES are significantly reduced. 46 (a) (b) (c) (d) Figure 4.1: SAR images obtained from the ERIM data by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES. CHAPTER 5 AMPLITUDE ESTIMATION 5.1 Introduction Consider the noise-corrupted observations of K complex-valued sinusoids [71] [74] K x(n) = Eak k' + v(n), n = 0,1,...N 1, (5.1) k=1 where ak denotes the complex amplitude of the kth sinusoid having frequency wk, N is the number of available data samples, and v(n) is the observation noise which is complex-valued and assumed to be stationary (and possibly colored) with zero-mean and finite unknown Power Spectral Density (PSD) q(w). We assume that {Wk =1 are known, with wk : w1, for k : 1. The problem of interest is to estimate {kkK=1 from the observations {x(n)}I-01. In this chapter, we describe a relatively large number of methods for solving this problem. Section 5.2 discusses least squares (LS) methods. LS methods are widely used for amplitude estimation because they are simple and easy to implement. If we restrict ourselves to estimating only one amplitude at a time, then the LS method reduces to the Discrete Fourier Transform (DFT) of the data at the frequency of the desired sinusoid and is computationally more efficient than the LS method that estimates K amplitudes simultaneously. Moreover, estimating one amplitude at a time does not necessarily require exact knowledge of the number of sinusoids in the data and of the frequency location of each sinusoid, which is a desired property in some applications. The disadvantage, however, is that using this one-at-a-time technique in general gives rather poor amplitude estimates when some sinusoids (that 47 48 are of interest to us) are close to one another. Statistical analyses that compare the merits of the two LS methods are also provided in Section 5.2. Since the LS methods completely ignore the correlation of the observation noise, they are in general suboptimal. By splitting the data vector into a number of subvectors, the covariance matrix of the noise-only part of the data subvectors can be estimated, which makes it possible to use a Markov-like estimator that is optimal in the class of Weighted Least Squares (WLS) techniques [68]. We describe in Section 5.3 several ways for estimating the aforementioned covariance matrix, which lead to different WLS amplitude estimators. Additionally, we show that, if the restriction of estimating one amplitude at a time is again imposed, we obtain two WLS amplitude estimators that are equivalent to the Capon [16] [44] and APES [52] methods extensively used for spectral analysis. The observation that some general spectral estimators, such as Capon and APES, can be used to solve the problem posed in (5.1) motivated us to seek other relatively sophisticated spectral analysis techniques for amplitude estimation. Both Capon and APES belong to the general class of filterbank approaches to spectral estimation [75], which involve splitting the data into subvectors, passing them through a set of narrowband filters (filterbank) whose center frequencies correspond to those that are of interest to us, and, finally, estimating the spectral density function at those frequencies from the filtered and, hopefully, signal-enhanced data. As one can see, the key issue of filterbank approaches is the design of the filters. A recent study has suggested the choice of matched filters, which gave rise to the MAtched-FIlterbank (MAFI) approach to spectral estimation [70]. Even though neither Capon nor APES was derived in the MAFI framework (see [16] and [52] for their original derivations), it was found that both are members of the MAFI approach [70]. In the light of the work of [70], we derive in Section 5.4 a generalized MAFI approach to amplitude estimation. Interestingly enough, we show that, under certain circumstances, MAFI 49 amplitude estimators have equivalent forms to the WLS methods. However, the MAFI approach is more general than the WLS technique in that the latter is a special case of the former. To show this, a new MAFI amplitude estimator that does not fall into the WLS category is described in Section 5.4. Other interesting MAFI amplitude estimators may exist and are yet to be discovered. A common feature of the amplitude estimators considered in this chapter is that none of them models the observation noise exactly. Even so, all methods are asymptotically statistically efficient, that is, they all achieve the Crambr-Rao Bound (CRB) in large samples. However, their finite-sample properties, which are of primary interest to this work, are quite different. Since the finite-sample analysis is intractable in most cases, we use Monte-Carlo simulations in Section 5.5 to compare these methods with one another. The amplitude estimation problem in (5.1) occurs in a variety of signal processing applications (see, e.g., [41] [55], and the references therein). In Section 5.6, we discuss its application to system identification. We show that, by using appropriate amplitude estimators, we can avoid the iterative search required by the standard system identification routines, such as the Output Error Method (OEM) [68], and achieve very good performance at a usually reduced computational load. In concluding this section, we introduce the following notation to distinguish among the various amplitude estimators. For instance, LSE(1, 0, 1) denotes the LS estimator that does not split the data (and hence it uses one data "snapshot"), uses no prefiltering, and estimates one amplitude at a time. Likewise, MAFI(L, K, K) denotes the MAFI estimator that splits the data into L subvectors, utilizes a bank of K prefilters, and estimates K amplitudes simultaneously. The remaining amplitude estimators are similarly designated. 50 5.2 LS Amplitude Estimators We consider two LS methods in this section, namely LSE(1, 0, K) and LSE(1, 0, 1). 5.2.1 LSE(1, 0, K) This is perhaps the most direct approach. Let us write the available data sequence in the following form x(0) 1 ... 1 cal v(0) x(1) ewl ... ei K a2 v(1) = + (5.2) x(N 1) ej(N-l1)w ... ei(N-1)K K v(N 1) or, with obvious definitions, x = Aa + v, (5.3) which is a linear regression equation. The LS estimate of a is & = (-AHA)-lAHx, (5.4) where (.)H denotes the conjugate transpose. Note that the noise is not modeled, even though it may be correlated. Despite this fact, LSE(1, 0, K) is asymptotically efficient [31]. A relatively simple manner to see this is as follows. First, note that E{&} = a, (5.5) where E{.} denotes the statistical expectation. The Mean Squared Error (MSE) of & is MSE{&} = cov{&}J E{( & )(& a)H = (AHA)- lAHWA(AHA)-, (5.6) where W E{vvH. (5.7) 51 Next, since (see, e.g., [34]) 1 lim I(AHA)= IN, (5.8) N-oo N where IN denotes the N x N identity matrix, and [(w1) 0 lim 1 lim N (AHWA)= [ .. (5.9) N--so N 0 O(WK) the asymptotic MSE is given by (W1) 0 lim NMSE{&} = ".. (5.10) N-+oo 0 O(WK) Under the mild assumption that v(n) is circularly symmetric Gaussian, the CRB for a is given by (see, e.g., [68]) CRB{a} = (AHW-1 )-1. (5.11) Using the following result (see [34] once again) lim -(AHW-1A) I (5.12) N->oo N 0 O-1(WK) we obtain (WP) 0 lim NCRB{a} = ] (5.13) N--+oo 0 O(WK) which coincides with (5.10). Remark: It can be readily checked from (5.6) and (5.11) that if v(n) is white, i.e. W IN, then LSE(1, 0, K) is statistically efficient for all N > K. 52 5.2.2 LSE(1, 0, 1) Since the observation noise v(n) is not modeled, an idea that reduces the computational burden quite a bit is to include K 1 sinusoids in the noise term, and hence estimate only one amplitude at a time. In some signal processing applications, the frequencies {Wk}kK=1 may be unknown. A typical way to estimate both {k} i=1 and {Wk K= would consist of estimating just one amplitude for varying frequency w and, then, detecting the peaks in the so-obtained spectrum [75] [41] [55]. As such, the assumption made in Section 5.1 that {Wk K=1 are known a priori may be relaxed when using the one-at-a-time technique. There is a somewhat subtle problem with the above technique: the sum of v(n) and K 1 sinusoids no longer has a finite PSD, and hence one of the previously made assumptions fails. Nevertheless, the idea still works as long as no two sinusoids (that are of interest) are spaced too close to one another, as shown below and later in Section 5.5. The LSE(1, 0, 1) is easily derived as N-1 k k= x(n)e-'"n k = 1,2,...,K, (5.14) n=o which is recognized as the DFT of {x(n)} Io' at Wk. The two estimates in (5.4) and (5.14) will be close to one another if Iwk W1 > 1/N (V k, 1; k $ 1) [75]. An analysis of LSE(1, 0, 1) runs as follows. Without loss of generality, let us consider (5.14) for k = 1. The LSE(1, 0, 1) estimate of al is given by & = ( ) -l Hx, (5.15) where = 1 ej ... eJ(N-1)wl (5.16) and where (.)T denotes the transpose. Taking the expectation of (5.15) yields 1 E{&I} = al + AHA, (5.17) N 53 where a = [ C2 ... aK ]T, and A is defined through [a A]A. (5.18) Hence, LSE(1, 0, 1) is biased. However, it is asymptotically unbiased (that is, its bias goes to zero as N -+ oo). We next calculate the MSE of &1: MSE{&,} = (.H ,)-lH (AaHAH + W) (dHi)-1. (5.19) Making use of (5.9) once again, along with the fact that AH&/v/N -- 0 as N -4 oc, we have lim NMSE{&1} = (wl). (5.20) N-oo Hence, LSE(1, 0, 1) is also asymptotically efficient. On the other hand, in finite samples (5.14) may be better or worse than (5.4), depending on the characteristics of the scenario under study. The fact that (5.4) may be better than (5.14) comes as no surprise. As an example, let us assume that the Signal-to-Noise Ratio (SNR) is high. Then, the bias of (5.14) dominates the variance part. On the other hand, (5.4) has no bias and its variance will be smaller than the bias of (5.14) if the SNR is large enough. Consequently, the MSE of (5.4) will be smaller than that of (5.14). The fact that (5.14) may be better than (5.4) is however a surprise. For an example of such a case, assume SNR < 1 and W = IN. Then, for (5.14), MSE {&I} (dH)-1, (5.21) whereas for (5.4), MSE({&} =[(AHA)-] (5.22) which can be much larger than (5.21) (e.g., if Iwk 1/N for some k > 2). In (5.22), []i,j denotes the ij-th element of the matrix argument. 54 Note that, for most cases of interest, LSE(1, 0, K) will give more accurate amplitude estimates than LSE(1, 0, 1), and that the difference between these two estimators is small for large N. On the other hand, LSE(1, 0, 1) is computationally more efficient than LSE(1, 0, K) since the matrix multiplication and inversion in (5.4) are avoided. Hence LSE(1, 0, 1) may still be worth considering. 5.3 WLS Amplitude Estimators If we split the data vector x into subvectors, then the covariance matrix of the noise part of the subvectors may be estimated and can hence be used to derive an optimal WLS estimator (i.e., a Markov-like estimator) [68]. In this section, we describe a number of such WLS estimators that split the data into vectors of shorter length, utilize no prefiltering, and estimate either one or K amplitudes at a time. 5.3.1 WLSE(L, 0,K) We define the following subvectors y(1)= x(1) x(l+1) ... x(l+M-1) l=0,1,...,L-1, (5.23) where L = N- M + 1. (5.24) The choice of M (M can be chosen smaller than K. See Figure 5.4. Moreover, when M = 1, all WLSE(L, 0, K) reduce to LSE(1, 0, K)) or, equivalently, of L is discussed in Section 5.5. We have 1 ... 1 ajlejwl v(l1) ejW ... ejdK 2ej21 v(l + 1) y(1) = + (5.25) ej(M-1)w1 ... e(M-)wK CKejwKl v(l + M 1) or, with obvious notation, y(l) = As(l) + e(l). (5.26) 55 Alternatively, we can rewrite (5.26) as y(1) = Ala + (1l), (5.27) where ejwll 0 Al A ADz. (5.28) 0 ewKl We will use (5.26) mostly for analysis and (5.27) for estimation. The WLS (Markov-like) estimate of a in (5.27) is given by L-1-1 L-1 a = [ A -lA A[H y(1)] (5.29) 1=0 l=0 where Q is an estimate of Q = E{e(1)e(1)}. (5.30) To estimate Q, we may proceed as follows. Let L-1 R = y ()yH (1). (5.31) l=0 One can verify that as L -4 oo, R goes to R = APAH + Q, (5.32) where P = ". (5.33) Hence, one way to estimate Q is as Q = A APAH, (5.34) where P is made from some initial estimates of {ak }K= obtained for instance via one of the LS amplitude estimators. The need for initial amplitude estimates is a 56 drawback of Q in (5.34). In the following we try to circumvent this need in two different ways. First, we show a way to simplify the WLSE(L, 0, K) that uses (5.29) with (5.34). From (5.34), we have that ](Q-A = APAH-1A + A = AF, (5.35) where Sr PAHQ-1A + IK. (5.36) For sufficiently large N and M, F is approximately diagonal since AHQ-1A is so (see, e.g., (5.12)). Consequently, (Q-Al = i-'AFD, Ri-1ADjr = R-tAr. (5.37) Inserting (5.37) into (5.29) yields (observe that TH cancels out) & [ f 1AH -I1A [ AH -ly(l) (5.38) 1=0 L=0 which, unlike using (5.29) with (5.34), does not require any initial estimate of {ak}k=1The amplitude estimator in (5.38) can be interpreted as an extension of the Capon algorithm in [16] [44] to multiple sinusoids. A different estimate of Q can be obtained as described next. Observe that K K APAH = Z[aka(wk)][aka(wk)]H k Z kH(, (5.39) k=1 k=1 where a(w) = 1 ejw ... ei(M-1)w (5.40) We can use the vectors {ikkK=1 introduced above to rewrite (5.27) as K y(l) = Z3ke k'+ E(l). (5.41) k=1 57 From (5.41), we can estimate 3k one at a time via LS as L-1 Sk = EY(e-jw," g(wk), k = 1,2,...,K. (5.42) l=0 (Note that we could estimate all {13k}k=1 simultaneously via LS, which however appears to perform even worse than using (5.42), especially for small N.) The use of (5.42) in (5.32) and (5.39) leads to the following estimate of Q K S= R g(wk)gH(wk). (5.43) k=1 The WLSE(L, 0, K) that uses (5.29) with (5.43) does not require any initial estimate of {ak }=1. It is an extension of the APES algorithm in [52] to multiple sinusoids with known frequencies. Remark: We note that E(k) and E(1) in (5.27) are correlated (for k 5 1), which implies that (5.29) is suboptimal (as it takes into account only the correlation between the elements of E(1), but ignores the correlation between E(1) and E(k), for k = 1). Yet, the WLS methods are likely to outperform the LS methods because the latter completely ignore the correlation in v(n). 5.3.2 WLSE(L, 0, 1) The particularization of WLSE(L, 0, K) to WLSE(L, 0, 1) is straightforward. Specifically, the WLSE(L, 0, 1) that corresponds to using (5.29) with (5.34) can be readily verified (by using the matrix inversion lemma) to be aH (Wk) f-1 (Wk) &k = k k = 1, 2,..., K, (5.44) aH(wk)if-a(Wk) whereas the WLSE(L, 0, 1) that corresponds to using (5.29) with (5.43) is given by aH(wk) g(wk)gH(wk) g(wk) &k = k = 1, 2,..., K. (5.45) aH(wk) [i g(wk)gH(k) a(wk) Note that (5.44), like (5.38), does not depend on P. However, unlike (5.38), the equation (5.44) is exactly equivalent to using (5.29) with (5.34). Equations (5.44) and 58 (5.45) are recognized to have the same form as the Capon [16] [44] and, respectively, the APES [52] spectral estimators. The two estimators were derived in [70] [47] by a different approach, namely the MAFI approach, which we will consider in a generalized form in the next section. It is interesting that the above two amplitude estimators, while both asymptotically efficient (and hence equivalent), have quite different finite-sample properties. Specifically, it was shown in [70] [47] that (5.44) is biased downward, whereas (5.45) is unbiased (within a second-order approximation) and in general has a better performance than the former. 5.4 MAFI Amplitude Estimators In this section, we derive a generalized MAFI approach to amplitude estimation. Let HH E CKxM be a matrix each row of which is a Finite Impulse Response (FIR) filter (for some 1 < K < M yet to be specified). The MAFI idea can be explained as follows: a) Design HH so that, when applied to {y(l)}, it maximizes the SNR at the K filter outputs. b) Estimate the amplitudes from the filtered data (whose SNR should be higher than that in the raw data) by, e.g., the LS or WLS technique. Mathematically, H can be obtained as follows: H = arg max tr [(HHQH)- HH(APAH)H (5.46) "Generalized SNR" where H is constrained in a way that is specified later (in particular, to guarantee that H is finite), and tr(-) denotes the trace of a matrix. Let XH = (HHQH)-1/2HHQ1/2, (5.47) 59 where (.)1/2 denote the Hermitian square root of the positive definite matrix argument. Observe that X is semi-unitary, i.e., XHX = IR. (5.48) The cost function in (5.46) can now be rewritten as f = tr [XH-1/2iAPAH -1/2X]. (5.49) It follows from the Poincar6 separation theorem (or the generalized Rayleigh quotient theorem) [37] that max f = Ak (Q-1/2APAH -1/2) (5.50) k=1 where {Ak(')}k=1 denote the eigenvalues of the matrix between the parentheses, ordered such that A1 > A2 > -... > AR; furthermore, the columns of the maximizing X are equal to the eigenvectors corresponding to {Ak k=1. Next, note that post-multiplying X by any unitary matrix of appropriate dimensions yields another valid solution for X. One such solution having a simple form can be obtained as follows. Observe that rank (^-1/2APAH -/2) = K, (5.51) which implies that we cannot improve the generalized SNR by choosing K > K since AK+1 = ... = Ag = 0. On the other hand, the larger the K the more filtered data will be available for amplitude estimation. Hence, we choose K = K. (5.52) In such a case, the maximizing X is given by X = Q-1/2AT, (5.53) 60 where T denotes some nonsingular matrix that makes X semi-unitary. One such T is T = (AHQ-A)-1/2. (5.54) Hence, X = Q-1/2A(AH -1A)-1/2 (5.55) We next observe that H = (-1/2X (5.56) satisfies (5.47). Consequently, we have H = Q-IA(AH -A)-1/2. (5.57) The final step is to observe that post-multiplying H by a nonsingular matrix does not change the generalized SNR criterion. Then, it follows immediately that H = Q(-'A(AHQ-'A)-' (5.58) maximizes the generalized SNR and it also satisfies the constraint HHA = IK. (5.59) The constraint (5.59) says that each (row) filter in HH passes one sinusoid undistorted, and completely annihilates the others. From (5.27) and (5.28), the filtered data corresponding to (5.58) is given by z(1) HHy(l) = Da + HHe(1) Dla + v(1), 1= 0,1,..., L 1. (5.60) The covariance matrix of v(1) can be estimated as HH QH = (AHQ-1A)-1. (5.61) It follows that the WLS (Markov-like) estimate of ca in (5.60) is given by L-1 -1 L-1 a = i DI(AH -1A)D] DH(AH -1A)(AHQ -A)-1AHQ-1 1=0 1=0 L-1 1 1 = E AH Q-1A, AHQ-ly(1) (5.62) l=0 1=0 61 which shows that MAFI(L, K, K) = WLSE(L, 0, K). (5.63) The MAFI interpretation of the WLS method, afforded by the above analysis, is interesting. In particular, it makes a clear connection between using the MAFI and the WLS techniques for amplitude estimation. The MAFI approach is however more general than the WLS technique. As an example, we derive a new MAFI amplitude estimator that does not belong to the WLS class as follows. Let zk(1) and vk(l) denote the k-th element of z(l) and, respectively, v(l) in (5.60). Then Zk(l) = ake' + vk(1), k = 1,2,...,K. (5.64) The above equations are related to one another only via the correlation between vk (1) and vp(l) (for k p). If we ignore the correlation, then the MAFI(L, K, 1) estimate of the ak via LS is given by 1L-1 k= L zk (l1)e (5.65) l=0 Unlike the Capon (5.44) and APES (5.45) estimators (which can also be shown to be members of the MAFI(L, K, 1) class [70]), the above MAFI(L, K, 1) estimator does require the knowledge of the number and frequencies of the sinusoids, which makes it behave more like a MAFI(L, K, K) estimator. In particular, it performs quite well for cases where some sinusoids are closely spaced, as will be seen in Section 5.5. Other interesting MAFI amplitude estimators may be devised by using some other choices of H in lieu of the one given in (5.58), as the solution to (5.46) is not unique. Specifically, one may introduce certain unitary transform on the H in (5.58), or choose K < K, or replace the IK in (5.59) by another nonsingular matrix, which all lead to solutions that are different from (5.58). Furthermore, one could even change the criterion in (5.46) to another reasonable definition of the "generalized SNR". However, such variations on the theme of MAFI are beyond the scope of the present work. 62 5.5 Numerical Examples In what follows, we investigate the performances of the various amplitude estimators described in the previous sections. For notational simplicity, we will refer to these methods as follows: LSE1: LSE(1, 0, 1) using (5.14); LSEK: LSE(1, 0, K) using (5.4); Capon1: WLSE(L, 0, 1) using (5.44); APES1: WLSE(L, 0, 1) using (5.45); CaponK: WLSE(L, 0, K) using (5.38); APESK: WLSE(L, 0, K) using (5.29) along with (5.43); MAFII: MAFI(L, K, 1) using (5.65) along with (5.43). We will compare these methods with one another as well as the CRB given in (5.11). Since all these methods are asymptotically efficient, we only consider the case when N is relatively small. Specifically, we choose N = 32. The data consist of three complex sinusoids corrupted by a complex Gaussian noise v(n) (to be specified): x(n) = s(n) + v(n), n= 0,1,...,N- 1, (5.66) where 3 s(n) = Z kej2xfk. (5.67) k=1 The frequencies of the sinusoids are fl = 0.1, f2 = 0.11, and f3 = 0.3. Also, cl = ejr/4, o2 ej/3, and a3 = ej/4. All examples are based on 200 Monte-Carlo simulations. The MSE figures shown in what follows are obtained as 1 200 MSE{jk} 200 &Ik(i) k 12, (5.68) i= 1 63 30 ........ I 20 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Frequency Figure 5.1: PSD of the test data that consist of three sinusoids and an AR(1) noise process. where ^kk(i) is the estimate of ak derived in the ith simulation run. 5.5.1 Estimation Performance versus SNR First, we consider the case where v(n) is colored. More exactly, v(n) is described by the following AutoRegressive (AR) process v(n) = 0.99v(n 1) + e(n), (5.69) with e(n) being a complex white Gaussian noise with zero-mean and variance O. The PSD of the test data is shown in Figure 5.1, where a2 = 0.01. The local SNR of the k-th sinusoid is defined as [41] lak 2 1 k 12 1SNR=1010g k+1/(2N) 10 10 (570) k-1(2N) Note the occurrence of N in the above SNR formula. For those methods that depend on M, we choose M = N/4 = 8, giving L = 25 (see Section 5.5.2 for a study of the effect of M on the performance). Figure 5.2(a) shows the MSEs of the seven amplitude estimators for 3, aloun.ng with the corresponding CRB, as the SNR varies. As one can see, APESc, APESK, and MAFI1 are very close to the CRB, while LSEK, which ignores the noise correlationh is evidently away from the CRB. CaponK also deviates from the CRB for relation, is evidently away from the CRB. CaponK also deviates from the CRB for 64 most SNRs. The reason is that the approximation made in (5.37) is valid only for large N and M, which is not the case in this example. Figure 5.2(a) also shows that both LSE1 and Capon1 are inconsistent (in SNR). Their inconsistency is not surprising because both are biased estimators. Recall that the bias of LSE1, as given in (5.17), does not vanish unless N goes to infinity. Similarly, Capon1 is always biased (downward) for finite N [70] [47]. Figure 5.2(b) shows the counterpart curves for al. (The results for a2 are omitted because they resemble those for al.) Note that f2 f1 = 0.01, which is smaller than 1/N c 0.03, the Fourier resolution limit. The performance degrades for all estimators under study, especially for LSE1, Capon1, and APES1, which estimate only one amplitude at a time. As shown in Figure 5.2(b), LSE1 and Capon1 essentially fail for all SNRs considered due to their large MSEs. APES1 is no longer close to the CRB but, unlike the previous two estimators, it still appears to be consistent (in SNR). As in Figure 5.2(a), CaponK again deviates away from the CRB at high SNRs. It appears that the approximation made in (5.37) introduces a bias (at small N and M) that may be negligible at low SNRs but dominates the variance at high SNRs. The bias does not disappear as the SNR increases, which causes the divergence of CaponK from the CRB. APESK performs quite well for high SNRs; however, it is not very stable at low SNRs (due to large variance). The best estimator in this example is MAFI1. The knowledge of the number and locations of the sinusoids, which the other one-at-a-time estimators may spare but is indispensable to MAFI1, appears to play an important role in its good performance in the current case that fails the other one-at-a-time estimators. As stated in Section 5.2, LSEK is statistically efficient, i.e. it achieves the CRB for any N > K, when the observation noise is white. To see how the other suboptimal (in finite samples) methods perform in such a case, we consider an example which is similar to the previous one except that v(n) is replaced by a zero-mean complex white 65 N=32,M=8,f3=0.3,colored Noise 10II 100 10 - LSE1 c -4 LSEK 10 ...... a Capon S x ....... x APES1 10 0o ..... CaponK S..... o APESK 10-6 +...... + MAFI1 CRB 10-7 20 25 30 35 40 45 50 Local SNR dB (a) N=32,M=8,f1=0.1 ,colored Noise 101 I 1 10 10 : v ... V..... ..... 13 ... 10-1 ......... . 0 ..... ......... -2 ..... ... ...... 10-2 ....... a -- LSE1 .. S10 C v v..... LSEK S......a Capon1 10 x....... APES1 S...... CaponK 10-5 ..... o APESK +...... + MAFI1 10-6 CRB 10-7 I I I I 15 20 25 30 35 40 Local SNR dB (b) Figure 5.2: Empirical MSEs and the CRB versus local SNR when N = 32, M = 8, and the observation noise is colored (an AR(1) process). (a) a3. (b) Cai. 66 Gaussian noise. The SNR is defined in the same manner as in (5.70). Figures 5.3(a) and 5.2(b) show the MSEs of the amplitude estimates of a3 and, respectively, al, and the corresponding CRB as the SNR increases. As one can see, the APES1, APESK, and MAFI1 estimates of a3 are again very close to the CRB; whereas for al, all suboptimal methods suffer from some performance loss as compared to the optimal LSEK, and yet the differences between LSEK and MAFI1 for all SNRs considered here are fairly small. A brief summary based on the previous study is as follows. APES1 is recommended in applications where it is known a priori that no two sinusoids are closely spaced (see, e.g., the application discussed in the next section), or when the closelyspaced sinusoids are of no interest. The reason to prefer APES1 to APESK or MAFI1 in such cases is that the former is more flexible than the latter two since APES1 does not necessarily require the knowledge of the sinusoidal frequencies. In terms of computational cost, APES1 and MAFI1 are similar to one another and both are simpler than APESK. When it is desired to estimate closely spaced sinusoids in colored noise, however, MAFI1 may be preferred. In general, we do not recommend the use of Caponl since it has a computational complexity similar to that of APES1 but is biased. Although we did notice that CaponK gives close-to-CRB performance at very low SNRs, in most cases of interest, other methods like APES1 or MAFI1 may be preferred. LSEK is statistically efficient and may be preferred when the observation noise is white; in cases where the white noise assumption is invalid, it is preferable to use APES1 or MAFI1. LSE1 gives comparatively rather poor estimation accuracy but is computationally quite simple. The performance differences stated so far occur only when N is relatively small. As N increases, all methods tend to the CRB, independent of the noise correlation. Hence, when N is sufficiently large, LSE1 should be preferred because of its computational simplicity. 67 N=32,M=8,f3=0.3,white Noise 10 I I 100 10-1 W10 -7-3 =10 - LSE1 S ..... v LSEK 10 .... a Capon1 S ....... x APES1 10- ..... CaponK e... o APESK 10-6 +...... + MAFI1 CRB 10-7 I I I I 20 25 30 35 40 45 Local SNR dB (a) N=32,M=8,f1=0.1,white Noise 101 I 10" ..... ........... . .. . .. ........... 100 .. 3 .. .. K ........ 10 - LSE1 ...... -4 .... .. 10 CapO w x ....... x APES1 10-s o ..... CaponK e .... o APESK + ...... MAFI1 10 CRB 10-7 I I 20 25 30 35 40 45 Local SNR dB (b) Figure 5.3: Empirical MSEs and the CRB versus local SNR when N = 32, M = 8, and the observation noise is white. (a) a3. (b) ao. 68 5.5.2 The Effect of M All WLS and MAFI amplitude estimators studied in this chapter depend on the choice of M, the subvector length. It is known that as M increases, all of them can better deal with the case of closely spaced sinusoids, but their statistical stability in general decreases [75]. Hence, there is a tradeoff to to kept in mind when choosing M. Note that M should also be smaller than N/2; otherwise, the estimated covariance matrix will be rank deficient. The following example examines the effect of M on the performances of these estimators. LSE1 and LSEK do not depend on M and are thus not considered in this example. The scenario is similar to the first example (AR noise) except that we fix a2 = 10-2, which corresponds to a local SNR of 30.8 dB for the first sinusoid (at fi = 0.1) and 39.2 dB for the third sinusoid (at f3 = 0.3). M is varied from 1 to 16 for all estimators except MAFI1, which requires that M > K (see (5.43) and (5.58)). The MSEs of the amplitude estimates of a3 and, respectively, al, and the corresponding CRBs are shown in Figures 5.4(a) and 5.4(b). As can be seen from these figures, all estimators are sensitive to the choice of M, to a smaller or larger extent. When no sinusoids are close to the one being estimated, such as the third sinusoid in this example, APES1, APESK, and MAFI1 perform quite well for a wide range of M. For the more difficult case as shown in Figure 5.4(b), the choice of M becomes very critical. Based on our empirical experience, a rule of thumb for the choice of M is given in Table 5.1. 5.6 Application to System Identification Consider the linear discrete-time system described by the following equation [68] x(n) = H(z-)u(n) + v(n), n = 0,1,..., N 1, (5.71) 69 N=32,a2=10-2,f 3=0.3,Colored Noise 101 I I I I S...... Capon1 10 x ....... x APES1 S...... CaponK o ...... o APESK 210- +.. + MAFI1 LL CRB 10-2 ..: as -2 " o 10 C103 -10-s. 0 2 4 6 8 10 12 14 16 Subvector Length M (a) N=32, 2=10-2 f=0.1,Colored Noise 104 I I I I 10 3 ......a Caponi x ....... APES1 102 ...... CaponK S...... o APESK .......+ + MAFI1 I 101 CRB = 10 ..x : 10 ..... ... 10 10-4 I I I I I I 0 2 4 6 8 10 12 14 16 Subvector Length M (b) Figure 5.4: Empirical MSEs and the CRB versus M when N = 32 and the observation noise is colored (an AR(1) process with a2 = 0.001). (a) C3. (b) c1. 70 Estimator M APES1 N/4 < M < N/2 APESK N/4 < M < N/3 MAFI1 N/8 < M < 2N/5 Capon1 or CaponK N/8 < M < N/4 Table 5.1: Choice of M for the WLS and MAFI amplitude estimators where the input u(n) is a sinusoidal (probing) signal K u(n)= ykeiwk", n= 0,1,...,N-1, (5.72) k=1 and the transfer function is rational: B(z-') blz-' +... + bz-q (5.73) A(z-1) 1 + alz-1 +... + apz-P' We assume that K > p + q. (5.74) Even if p and q were unknown, K could still be chosen sufficiently large to satisfy (5.74). The problem of interest in this section is to estimate {ai}lP and {bj},=l from {x(n) }yo'5.6.1 System Identification Using Amplitude Estimation The commonly-used Output Error Method (OEM) does not model v(n) and obtains estimates of {ai} ,1 and {bj}j=l by minimizing the criterion N-I COEM(a, b) = E Ix(n) H(z-')u(n)l2, (5.75) n=o wherea=[ al ... ap ]T andb= [ bi ... bq ]T. Let ak(a, b) = ykH(e'"k). (5.76) 71 For sufficiently large N (so that the transient response in the output can be neglected), the cost function COEM(a, b) is approximately equivalent to N-1 K 2 C, (a, b) = E x(n) E Ok(a, b) e4kn (5.77) n=0 k=1 The method that we propose for estimating a and b is based on (5.77) and consists of two steps: First estimate {ak }=i in an unstructured/non-parametric form. Then fit {ak(a, b)}K=1 to the amplitude estimates obtained in the previous step by taking into account the statistical variance of the latter. In what follows, we detail the above two steps. Step 1: Use an appropriate amplitude estimator to obtain estimates {&k k=1 of {ak kK=1 from the measurements {x(n)}N=01.. APES1 may be recommended in this case because we have control over the probing signal and we have no reason to choose any of the sinusoids too close to one another. The large-sample variance of the estimated amplitudes {&k JK=1 is proportional to {k(wk)}K=1 (see Section 5.2). To obtain estimates of {(Wk)}K=1, we can first calculate K i(n) = x(n) ,&k kn, n= O,,...,N-1, (5.78) k=1 and then utilize either a parametric or a non-parametric PSD estimator [75] [41] on (5.78) to obtain {(Wk)kK=1. Specifically, in the examples given in Section 5.6.2 we use the Capon PSD estimator [16] [44] [75] (also see Section 6.3), which determines { (Wk) K=1 as M (Wk) = k = 1, 2,..., K, (5.79) aH(wk)ii a(Wk)' where a(wk) is defined in (5.40) and R, is the sample covariance matrix of the estimated noise vectors v )= I(l) (+1)... (+M-1) = 0,1...L 1, (5.80) 72 that is, L-1 L1 (5.81) Step 2: Obtain estimates of {ai, bj} by minimizing K C2 (a, b) = E &k ak(a,b)12. (5.82) k=1 O(Wk) To do so we can use a host of methods, provided that we have good initial estimates of a and b. To obtain such estimates and then minimize (5.82), we assume that p and q are known. (Standard techniques for system order determination can be found in, e.g., [68] [54].) We pick up the p+q largest {&k} (if the SNR is low, an alternative is to choose those {&k} that have the largest ratio &k 12 (Wk), assuming that (wk) was estimated) and define a criterion made from the corresponding terms of (5.82) P+q C3(a, b) = E I&k ak(a, b)12 (5.83) k=1 O(Wk) where we have assumed, for notational simplicity, that {&k} }(i are the p + q chosen amplitudes. Now, the minimization of (5.83) is simple. Indeed, almost always one can choose a and b to satisfy &k = Ok(a,b), k = 1, 2,..., p + q. (5.84) Equation (5.84) is equivalent to GkA(eJWk) = B(ejwk), k = 1,2,... ,p + q, (5.85) 'Yk which can be rewritten as a linear system of p + q equations with p + q unknowns {ai, bj}. That system will generally have a unique solution that makes (5.83) equal to zero, and which therefore gives our initial estimates of {ai, bj}. As shown in the following numerical examples, the initial estimates are usually quite good. Hence, one can even skip the step of minimizing (5.82) to save computations. 73 Remark: According to the Extended Invariance Principle (EXIP) [76], the estimates of {ai, bj} obtained by minimizing (5.82) achieve the CRB asymptotically, and hence they have a better asymptotic accuracy than the OEM estimates whenever v(n) is colored. It also follows from this observation that in the case of K = p + q, the estimates obtained from (5.85) are asymptotically efficient. This latter result (of a somewhat limited interest, due to the requirement that K = p + q) was first proved in [40] in a much more complicated way. 5.6.2 Numerical Examples The following examples assume that p and q are known to facilitate performance comparison. It is reasonable to do so since both OEM and the proposed method use similar techniques to determine the model orders. Also, we adopt the strategy to choose the p + q largest {rk} in Step 2 of the proposed method. Example 5.6.2.1 The system considered in this example is given by (5.73) with A(z-1) = 1 1.6019z-1 + 0.9801z-2, (5.86) and B(z-1) = z-1 + 0.2472z-2 + 0.1600z -3. (5.87) The probing signal is given by u(n) =2 cos(2r0.05n) + 2 cos(2w0.15n) + 2 cos(270.25n) +2cos(27r0.35n) + 2cos(2rO0.45n), n = 0,1,...,N 1. (5.88) We consider using a real-valued probing signal because this is the usual case in practice. (A subtle question arises as the amplitude estimation techniques discussed in the previous sections all assume that the sinusoids are complex-valued. One might impose certain conjugate symmetry constraint and derive similar techniques that are specifically tailored for real-valued sinusoidal amplitude estimation so that, if 74 w1 = -w2, the estimators will give &i = &]. Yet, our experience shows that the gain would most often be minor and hence the effort is not worthwhile. See [39], for example.) Note that K = 10 for this case. The noise v(n) is a real-valued white Gaussian noise with zero-mean and variance o2 = 0.01. We estimate the system parameters using the proposed technique and OEM (OEM is provided in the System Identification Toolbox of MATLAB). For the proposed technique, we compute both the initial estimates given by solving (5.85) and the minimizer of (5.82). The minimizer of (5.82) is found by using the solution of (5.85), obtained by APES1, as the initial condition and then evoking a standard gradient-type nonlinear optimization routine provided by MATLAB. To reduce the number of graphs, we only show the averaged Root Mean Squared Error (RMSE) for the a-parameters RMSE{i} = E RMSE{&i} (5.89) i=1 and similarly for the b-parameters. All results are based on 200 Monte-Carlo simulations. Figures 5.5(a) and 5.5(b) show the averaged RMSEs of the a-parameters and, respectively, the b-parameters obtained by using OEM and the proposed technique, as N increases. Figure 5.5(c) shows the required number of flops as N increases. (APES1 and MAFI1 uses M = 20 in this and the following example, which does not fall in the range given in Table 5.1. The reason is that APES1 or MAFI1 with M = 20 is quite acceptable for the probing signal in (5.88) that contains well-separated sinusoids and, moreover, choosing a larger M would result in additional computations.) Finding the minimizer of (5.82) or the OEM estimates involve iterative searches which give variable flop counts from trial to trial. The number of flops needed by each of these two methods, as shown in Figure 5.5(c), is the average over 200 trials. As one can see, the initial estimates of {ai, bj} given by solving (5.85) with APES1 or MAFI1 have similar RMSEs to those obtained by OEM. The estimates obtained by minimizing (5.82) are slightly better than the initial estimates obtained by APES1 75 or MAFI1, but at a significantly increased computational cost. Due to this observation, we do not recommend using this approach, i.e., minimizing (5.82), for refined estimation accuracy. Other more sophisticated techniques for system identification (see, e.g., [68] [54]) may be preferred in that event. Figure 5.5(c) also shows that, as compared to OEM, there is little computational advantage of using the initial estimates obtained by APES1 and MAFI1. The reason may be that the system in this example is quite simple (it has white output errors, etc.) and, apparently, OEM reaches convergence in a relatively small number of iterations. For a more complex system, such as the one used in the next example, OEM may need more iterations to converge. It should be mentioned that we did not program our method very carefully and hence our code is unlikely to be as efficient as the OEM code in MATLAB. Regarding the estimation accuracy, we shall stress that in the current case where the noise v(n) is white, OEM coincides with the optimal Maximum Likelihood Method (MLM) [68] [54]. When v(n) is colored, OEM is no longer MLM. In that case, the initial system parameter estimates obtained by APES1 or MAFI1 may outperform those by OEM, as in fact shown in the next example. Recall that LSEK is statistically efficient when the observation noise is white. Then, one might wonder why the initial estimates given by LSEK may be notably worse in such a case than those given by APES1 or MAFI1, as happened in the previous example (especially when N is small). The reason is that the transient response of this system cannot be neglected for small N. To show this, the PSD of x(n) is estimated by using the Capon PSD estimator, with N = 200 and M = 20, and is plotted in Figure 5.6. It shows two extra peaks (which behave like two sinusoids) at 0.1. The extra peaks are attributed to the response of the system (which has poles at 0.99ej2r0.1) to the initial conditions. Since it is essential for LSEK to have the accurate knowledge of the number and frequencies of the sinusoids frequencies to give reliable amplitude estimates, its performance in the previous example is considerably 76 x 10- a-Parameters b-Parameters (85) with LSEK 7 v (85) with LSEK 0.07- (85) with APES1 S. (85) with APES1 0 (85) wth MAFI 6 V o a (85) with MAFi 0.06 (82) with APES1 (82) with APES1 OEM U0 5- OEM 0.05 4 0.04 0.02 1 0 .0 1 -* ,... V ... 10 10 10 10 Data Length N Data Length N (a) (b) Computational Complexity S v (85) with LSEK (85) with APES1 ac (85) with MAFI1 4 4 (82) with APES1 2 ......o. 1 0 s 10 10 Data Length N (c) Figure 5.5: Averaged RMSEs and the number of flops versus N for the first system when the observation noise is white (a.2 = 0.01) and M = 20 for APES1 and MAFI1. (a) RMSE of a-parameters. (b) RMSE of b-parameters. (c) Number of flops. 77 I I i I I 15 Ca 10 -20 I I I I I I I I -.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Frequency Figure 5.6: PSD estimate of the output of the first system corrupted by white noise with a2 = 0.01 and N = 200. deteriorated. On the other hand, the above knowledge is not necessarily needed by APES1 and hence its performance is not affected. Unlike APES1, MAFIl does require this knowledge. Yet, the initial condition response is substantially weakened through the frequency selective filtering employed by MAFI1 and hence has little, if any, effect on the amplitude estimates and the system parameter estimates. As N increases, the transient effect becomes less severe, and, consequently, the initial system parameter estimates obtained by using LSEK approach those obtained by APES1 or MAFI1. Example 5.6.2.2 We now consider a second system with A(z-1) = 1 1.9109z-1 + 1.7251z-2 0.7033z-3 + 0.2450z-4, (5.90) and B(z-1) = z-1 + 1.0562z-2 + 0.6100z-3 + 0.1912z-4 + 0.0400z-5. (5.91) The noise v(n) is an AR(1) signal as in (5.69) except that e(n) is now replaced by a real-valued white Gaussian noise with zero-mean and variance a2 = 0.01. The probing signal is the same as in the previous example. Figures 5.7(a) to 5.7(c) show the averaged RMSEs of the a-parameters and the b-parameters, as well as the number 78 a-Parameters b-Parameters 0.7 0.9 v (85) with LSEK v (85) with LSEK 0.6 ........ (85) with APES1 I0.8 (85) with APES1 o o (85) with MAFIl I ao o (85) with MAFI1 (82) with APES1 0.7 o. (82) with APES1 0.5- OEM - OEM L PEM 0.6 PEM 0.4 0. <> 0.3 0.2 0.2 0 .1 ".... . 10 103 10 10, Data Length N Data Length N (a) (b) Computational Complexity 10 ............. .. E V .... . S v v... (85) with LSEK ........ (85) with APES1 10s- o (85) with MAFI1 0 (82) with APES1 OEM o a PEM 104 10 10a Data Length N (c) Figure 5.7: Averaged RMSEs and the number of flops versus N for the second system when the observation noise is colored (an AR(1) process with .2 = 0.01) and M = 20 for APES1 and MAFI1. (a) RMSE of a-parameters. (b) RMSE of b-parameters. (c) Number of flops. of flops, as N increases. As one can see, the initial system parameter estimates given by APES1 or MAFI1 are significantly better than those given by OEM, and yet the former two are computationally more efficient that the latter. 5.7 Summary This chapter examines the problem of amplitude estimation of sinusoidal signals in colored noise. Three general classes of estimators, namely the LS, WLS, and MAFI approaches to amplitude estimation, are discussed. It is shown that, under 79 certain circumstances, the MAFI approach to amplitude estimation is equivalent to the WLS approach, and yet the former is more general and includes the latter as a special case. The amplitude estimators under discussion can be further categorized depending on whether they estimate one amplitude at a time or all amplitudes simultaneously. MAFI or WLS methods, such as APES1, APESK, and MAFII, in general give more accurate amplitude estimates for sinusoids in colored noise. Methods that estimate only one amplitude at a time, such as APES1, mostly do not require the exact knowledge of the number and locations of the sinusoids and hence are more robust than those that estimate all amplitudes simultaneously. As an application example, a system identification application using sinusoidal probing signals is discussed. A new technique for system identification is presented that can avoid iterative searches through fitting the system parameters to the output amplitude estimates. It is shown that, by using this technique with appropriate amplitude estimators, such as APES1 or MAFI1, we can obtain results that are generally better than those corresponding to the widely-used iterative OEM, yet usually at a reduced computational cost. CHAPTER 6 CAPON ESTIMATION OF COVARIANCE SEQUENCES 6.1 Introduction Covariance sequence estimation is a ubiquitous task in digital signal processing. A standard technique for estimating the covariance sequences is the so-called the standard sample covariance estimator. The standard covariance estimates are consistent provided that the given signals are ergodic to the second order. However, there is a major concern of using the standard estimator due to the unrealistic windowing of the observed data it assumes; that is, it assumes that the data beyond the observed duration either is zero or repeats itself periodically. Partly for this reason, there have been several attempts in the literature to derive more accurate covariance estimates than the standard ones. A notable example is the approach based on the Burg autoregressive (AR) spectral estimator. However, the so-obtained covariance estimator was found even less accurate than the standard sample covariance estimator [78]. To be more exact, the Burg approach was shown to have larger variances than the standard method. Another approach, which has generated a whole new research direction, relies on the maximum likelihood (ML) principle [13]. However, the ML estimation of covariance sequences is a computationally involved problem which does not have a closed-form solution. The solution given in [13] is iterative and not guaranteed to be globally optimum. Apparently, there exist no compelling alternatives to the standard method that can be recommended for general use. In this chapter we present a new method, namely the Capon method [48], for covariance sequence estimation. The Capon method obtains the covariance sequence estimates by Fourier inverting the Capon power spectral density (PSD) estimates. 80 81 There are basically two Capon PSD estimators, referred to as Capon-1 [16] [44] and Capon-2 [45] herein. We find that, while Capon-2 is capable of finer spectral resolution around the peaks of a spectrum, it is generally a globally poorer spectral estimator than Capon-1. We hence concentrate our interest on Capon-1 for covariance sequence estimation in this chapter. Since the Capon spectra, i.e., the PSD estimates, are shown to be equivalent to AR or autoregressive moving-average (ARMA) spectra, the inversion procedure for computing the exact covariance sequences corresponding to the Capon spectra can be implemented in a rather convenient way. (Note that the calculation of the covariance sequences corresponding to the Capon spectra is an interesting problem by itself.) We also present an FFT-based approximate method to compute the covariance sequences from the Capon spectra. It has been found that the approximate method provides covariance estimates that are almost identical with those obtained by the exact method, while the computational complexity is greatly reduced. Our primary interest is to apply the Capon method as well as the standard approach to ARMA signals. To that end, a few ARMA signals with typical pole and zero locations are studied in our numerical examples. The studies show that considerable improvements are attained by the new Capon method. The Capon covariance estimation method can be readily used in many applications. One important class is the ARMA spectral estimation. Since most ARMA spectral estimators rely on the Yule-Walker equations to determine the AR coefficients, it may be expected that the better the covariance estimates used, the more accurate the AR coefficient estimates yielded. We examine how the Capon covariance estimates can be used with the overdetermined modified Yule-Walker (OMYW) method [14] [69] to compute more accurate AR coefficients. We find that the performances of the usage are influenced by the pole and zero locations and, still, generally 82 better AR coefficient estimates are obtained by using the Capon covariance estimates than by the standard ones. Another application discussed in this chapter is the moving-average (MA) model order determination by making use of the Capon and the sample covariance estimates, where we find that better performance is achieved by the former. It should be mentioned that we can obtain APES covariance sequence estimates similarly by Fourier inverting the APES PSD estimates. However, in spite of the evident difference of the Capon and APES complex spectral estimates for discrete spectral components, their performances in continuous PSD estimation are similar to one another, which implies that the APES covariance estimates are similar to the Capon covariance estimates. As such the APES covariance estimation is not discussed herein. 6.2 Standard Covariance Estimator and Outlook With no other assumptions made on the signal under study, except for assuming the second order ergodicity, there are two ways to obtain the standard sample covariances of the signal, namely, the biased and the unbiased covariance estimators. However, the biased covariance estimator is more commonly used since it provides smaller mean-squared errors (MSE) than the unbiased one and guarantees the covariance estimates to be positive semidefinite [75]. The biased sample covariance estimator of a wide-sense stationary signal with zero-mean has the form N-k f(k)= y=Y*(n)y(n+k), k=0,1,...,K. (6.1) n=l where {y(n)}N1 are the observed data samples, N is the number of samples, f(k) denotes the estimate of the covariance function r(k), K is the largest lag desired (0 < K < N 1), and (.)* denotes the complex conjugate. Note that (6.1) is asymptotically unbiased. 83 The estimator given in (6.1) is consistent if the ergodicity assumption is satisfied. A study on whether the standard sample covariance estimator is also asymptotically statistically efficient, i.e., whether it asymptotically achieves the Crambr-Rao Bound (CRB), has been undertaken in [60]. Let y(t) be an ARMA(p, q) signal. If p q, then the sample covariance estimate i(k) is asymptotically statistically efficient if and only if 0 < k < p q; in particular, for AR processes of order p, f(k) is asymptotically efficient for 0 < k < p, but inefficient for all other k. If p < q, none of ?(k) is asymptotically efficient; in particular, none of ?(k) is asymptotically efficient for an MA process. It is known that the sample covariance sequence {f(0), ..., (N 1), 0, 0,...) and the data periodogram constitute a Fourier transform pair. It is also known that the periodogram is a statistically inefficient (in fact inconsistent) estimator of the PSD [75]. This observation suggests that better covariance estimators might be obtained by Fourier inverting better PSD estimators. However, this is not necessarily so. Briefly stated, the reason is that the Fourier transform and the inverse Fourier transform are integral transforms and hence small errors in one domain may be associated with large errors in the other domain or vice versa. The fact that the covariance estimates in (6.1) are consistent whereas the periodogram is not illustrates this observation. Also note that, while the Burg estimate of the PSD is typically more accurate than the periodogram, the corresponding Burg estimate of the covariance sequence is generally poorer than (6.1) [78]. In spite of the fact briefly discussed above, in the following we consider estimating the covariance sequences by inverting a PSD estimate that is often much more accurate than the periodogram, namely the Capon PSD estimate. Like the periodogram, no model is assumed in the Capon PSD estimator, which makes it more robust than the parametric estimators in many situations. Although it has lower spectral resolution than the AR spectral estimator, it generally exhibits less variance 84 than the latter [44] [75]. Another reason that we consider the Capon PSD estimator is that it does not exhibit the so-called correlation matching property [75] [41]; that is, the inverse Fourier transform of the Capon PSD estimates does not yield the same covariance sequences used to obtain the Capon PSD estimates. This fact allows us to obtain a new covariance estimator from the Capon spectra. 6.3 Capon PSD Estimator We have derived in Section 3.2.1 the Capon filters and Capon amplitude and phase spectra. In this section, we derive expression for the Capon PSD estimates. The Capon filter is rewritten below for easy reference (see (3.28)): hCapon R- (w) (62) S aH (w)RLaM am. The filter output power is given by E IhH(l) = hHRh 1 (6.3) SaH(w)RaM( (6.3)) Let 3 denote the bandwidth of the filter given by (6.2). Then the Capon PSD estimate has the form E { hHf(l)2} 1 (w) =. (6.4) Since the (equivalent) time-bandwidth product is equal to unity, one way is to choose 3 as the reciprocal of the temporal length of the Capon filter; that is 1 0 = 1. (6.5) M By choosing the filter bandwidth as given by (6.5), we obtain the so-called Capon-1 PSD estimator [44] [75]: Capon-1: ((w aM(W (6.6) where we have replaced R by the sample covariance matrix fR. We may use the forward-only sample covariance matrix, but we prefer using the forward-backward 85 sample covariance matrix, given in (3.13), for better statistical properties associates with the latter [41]. Another more elaborate choice of / is obtained as the equivalent bandwidth of IH(w)12, where H(w) is the filter's frequency response: H(w) [hCapon HaM(W). (6.7) This specific bandwidth choice leads to the Capon-2 PSD estimator [45] [75]: Capon-2: O(w) = a I-M (6.8) aH(w)R-2aM (w) Burg showed that the inverse of Capon-1 spectrum is equal to the average of the inverses of the estimated AR spectra of orders from 0 to M [11]. This observation reveals the fact that Capon-1 has less statistical variation as well as lower spectral resolution than the AR estimator. A similar but more involved relationship between Capon-2 and the AR estimators was derived in [75]. Theoretically, the performance of Capon-2 is hard to quantify. However, it is generally believed that Capon-2 possesses finer resolution and hence is a better spectral estimator than Capon-1 [45]. We will show here, with a typical example, that even though Capon-2 has better resolution locally around the power peaks, it is globally a more biased estimator than Capon1. Our experience also shows that Capon-2 generally gives much poorer covariance estimates than Capon-1. Therefore, Capon-2 is not recommended for covariance sequence estimation. To illustrate the above claim, consider an ARMA(4,2) signal y(n) = 2.76y(n 1) 3.809y(n 2) + 2.654y(n 3) 0.924y(n 4) +e(n) 0.9e(n 1) + 0.81e(n 2), (6.9) where e(n) is a real white Gaussian random process with zero-mean and unit variance. The Capon-i and Capon-2 spectral estimates with N = 256 and M = 50 are shown in Figure 6.1(a), where the dashed curve stands for the true PSD of the ARMA 86 ARMA(4,2)(N=256, M=51, 100 realizations.) ARMA(4,2)(N=32, M=1 1, 100 realizations.) 0 40 plots are the averages of 100 independent realizations. (a) N = 256, M = 50; (b) N 32, M = 10. and Capon-2 estimates. It is obvious that Capon-2 is highly biased and suffers from 0 0.1 0.2 0.3 0. 0.5 0 0.1 0.2 0.3 0.4 0.5 Frequency Frequency (a) (b) Figure 6.1: Power spectral density estimates by using Capon-i and Capon-2. The plots are the averages of 100 independent rer of the overal) N = 256,M = 50; (b) N = 32,M = 10. signal, while the solid and the dashdotted lines, respectively, indicate the Caponand Capon-2 estimates. It is obvious that Capon-2 is highly biased and suffers from a significant power loss. This observation has also been made with other signals, especially if the signal is a narrowband signal. An explanation of this behavior follows. The calculation of the filter bandwidth in Capon-2 is applicable only if the Capon filter is a narrowband filter. Recall that the Capon method aims to find the Capon filter that minimizes the total output power of the overall frequency band while passes the current frequency w undistorted. No effort has been taken to make sure that the Capon filter is narrowband. Let the lobe of the Capon filter frequency response where the current frequency of interest w is located be called as the mainlobe, while all the others are called the sidelobes. It has been found that the steering frequency w is not necessarily at the maximum or the center of the mainlobe [75] [52]. Furthermore, if the input signal is a narrowband signal, there may exist "sidelobes", located at frequency bands where the power level of the input signal is low, that are even larger than the "mainlobe" of the Capon filter frequency response. Note 87 that the large "sidelobes" do not make any significant contributions to the filter output power so that the filter design criterion is still satisfied; that is, the output power is minimized, while the frequency response at w is one. In all such cases, the Capon filter is not a narrowband filter and hence it calculates an overestimated filter bandwidth. Hence the Capon-2 PSD estimates become highly biased. However, it is interesting to note that Capon-2 does possess higher resolution capability, around the power peaks, than Capon-1. This is illustrated in Figure 6.1(b), which shows the PSD estimates of the same ARMA signal as used in Figure 6.1(a) but with N = 32 and M = 10. The Capon-1 estimator cannot resolve the two power peaks this time, while Capon-2, albeit biased, still can. For the preceding reasons we do not consider using Capon-2 for covariance sequence estimation in the sequel. 6.4 Capon Covariance Estimator We describe below how the Capon PSD estimates can be Fourier inverted in a rather convenient manner yielding the Capon covariance sequence estimates. The study of the covariance sequences corresponding to the Capon spectra is an interesting endeavor by itself, which apparently has not been undertaken in the literature before. We also present an approximate but computationally more efficient method to calculate the Capon covariance estimates from the Capon PSD estimates. 6.4.1 Exact Method Theorem 6.4.1 Let F = {fi,j} C(M+1)x(M+1) and let aM(w) 1 e ... ej(M-1) (6.10) then M-1 a (w)raM(w) seJ, (6.11) s=-(M-1) 88 where min(M+s-1,M-1) Is = rk,k-s. (6.12) k=max(0,s) If r is Hermitian, then pL = p*,. Proof: See Appendix D. It is obvious that whenever P is non-negative definite we have aH (w)FaM(w) > 0 for any w. Thus aHM(w)aM(w) is a valid power spectrum. Furthermore, Theorem 6.4.1 indicates that in such a case 1/aH(w)raM(w) is in fact the power spectrum of an (M 1)th-order AR process. Consequently, Capon-1 yields an equivalent AR(M 1) process (whereas Capon-2 yields an equivalent ARMA(M 1, M 1) process). By making use of (6.12), we can find the coefficients of the equivalent AR process. The calculation of the exact covariance sequences from the AR coefficients is a standard problem and can be solved, for example, via the inverse Levinson-Durbin algorithm (See [75] [41] and the references therein for more details). Hence the implementation of the Capon method for covariance estimation runs as outlined below: Step 1: Pick up a value for M(M < N/2) and compute I by (3.13). Step 2: Compute p, associated with P = -1 by (6.12). Factorize M-1 m=-(M-1) (say, by using the Newton-Raphson algorithm) and obtain the (minimum-phase) spectral factor. Step 3: Compute the corresponding covariance sequence f{temp(k) } from the spectral factor (or, equivalently, the AR model) by, for example, the inverse Levinson-Durbin algorithm. While using the Fourier inverting method for covariance estimation, it is necessary that the integral of the PSD estimate over all frequencies gives a good estimate of the signal power; otherwise there may be scaling errors in the covariance estimates. |

Full Text |

74 l\ CO2, the estimators will give on = Yet, our experience shows that the gain would most often be minor and hence the effort is not worthwhile. See [39], for example.) Note that K 10 for this case. The noise v(n) is a real-valued white Gaussian noise with zero-mean and variance er2 = 0.01. We estimate the system parameters using the proposed technique and OEM (OEM is provided in the System Identification Toolbox of MATLAB). For the proposed technique, we compute both the initial estimates given by solving (5.85) and the minimizer of (5.82). The minimizer of (5.82) is found by using the solution of (5.85), obtained by APESl, as the initial condition and then evoking a standard gradient-type nonlinear optimization routine provided by MATLAB. To reduce the number of graphs, we only show the averaged Root Mean Squared Error (RMSE) for the a-parameters (5.89) i= 1 and similarly for the -parameters. All results are based on 200 Monte-Carlo simula tions. Figures 5.5(a) and 5.5(b) show the averaged RMSEs of the a-parameters and, respectively, the -parameters obtained by using OEM and the proposed technique, as N increases. Figure 5.5(c) shows the required number of flops as N increases. (APESl and MAFI1 uses M = 20 in this and the following example, which does not fall in the range given in Table 5.1. The reason is that APESl or MAFI1 with M = 20 is quite acceptable for the probing signal in (5.88) that contains well-separated sinu soids and, moreover, choosing a larger M would result in additional computations.) Finding the minimizer of (5.82) or the OEM estimates involve iterative searches which give variable flop counts from trial to trial. The number of flops needed by each of these two methods, as shown in Figure 5.5(c), is the average over 200 trials. As one can see, the initial estimates of {a,j} given by solving (5.85) with APESl or MAFI1 have similar RMSEs to those obtained by OEM. The estimates obtained by minimizing (5.82) are slightly better than the initial estimates obtained by APESl 42 and Q 'aw,, \H .H T} 1 m2 Mi ,M2 aMi,M2R aMuM2 4 LxL2 V^R-1^ = iibK,^)f+lbT("1-1J2)e^'^)f. 11 ^ 2j" L1L2-||e(a;1,a,2)||2 (4.20) Next we observe that b(wi,u;2), d(u>i, c FFT. Specifically, we partition C_1, D and E as follows: C_1 = vec{Bi} ... vec{BMlM2} and D E ~\ T vec{Di} ... vec{DMlM2} iT (4.21) (4.22) (4.23) vec{E!} ... vec{EMlM2} where B*, G CMlxM2, CLlXl2 and E*, G CilX2. Then (again, we drop the dependence on ux and o>2) (aM2 aM JvecjB!} ... (a^r <8> )vec{BMlA2} aMi-^laM2 aM1BM1M2a*M2 (4.24) Likewise, we have and d = Dai1|La e = Ea *LlM a^Dia l2 a1DMiM2al2 afEia. ... a? Ej - - (4.25) (4.26) 1Li-Eji1l2 aLih;M1M2aL2 Note that a^B^a^ represents the 2-D discrete Fourier transform of Bfc at (a>i,a;2). Hence APES can be efficiently implemented by first calculating b(u>i,u;2), d(cji,o>2) 79 certain circumstances, the MAFI approach to amplitude estimation is equivalent to the WLS approach, and yet the former is more general and includes the latter as a special case. The amplitude estimators under discussion can be further categorized depending on whether they estimate one amplitude at a time or all amplitudes simul taneously. MAFI or WLS methods, such as APES1, APESK, and MAFI1, in general give more accurate amplitude estimates for sinusoids in colored noise. Methods that estimate only one amplitude at a time, such as APESl, mostly do not require the exact knowledge of the number and locations of the sinusoids and hence are more robust than those that estimate all amplitudes simultaneously. As an application example, a system identification application using sinusoidal probing signals is discussed. A new technique for system identification is presented that can avoid iterative searches through fitting the system parameters to the output amplitude estimates. It is shown that, by using this technique with appropriate amplitude estimators, such as APESl or MAFI1, we can obtain results that are generally better than those corresponding to the widely-used iterative OEM, yet usually at a reduced computational cost. 109 where {Re(z)} is the covariance sequence of ew(/) or ew(/). Note that (B.7) and (B.8) are due to the circularly symmetric distribution assumption. It then follows from (B.9), (B.10), and Condition C that: OO lim Le\sSH\ = lim Le{~S~SH\ = V Re{i)e-juji = Ltoo L J Lt oo l J z' ioo where the last equality follows from the standard results on the transfer of spectral densities through linear systems. Among others, the previous calculations imply that, as L oo,g and g tend to aa^ and oOlm (in the mean square sense), respectively, and, therefore, GGfi goes to \oi\2aLM&M- Hence hBBC and hBBA have the same limit as L oo. Let h denote a generic FIR vector and let denote the deterministic vector that is the limit of (the possibly random) h when L goes to infinity. Observe that for all methods under study, the associated h and hoo vectors satisfy hHaM = 1, and h^,aM = 1. (B.12) By using this observation with (3.26) and (B.l), we obtain = hHg = a + hHS. (B.13) Since 5 tends to zero as L goes to infinity, it follows from (B.13) that the estimation error can be asymptotically written as (to a first-order approximation) a. a ~ h^d. (B-14) Then it readily follows that LE {(a a)2} 0, as L oo, (B.15) and lim LE {\a- o:|2} = h^, lim LE {<5<5/7) = <Â£e(cj) |h^aM|2 = 0e(w), (B.16) L-* oo L>oo L J 11 and the proof is concluded. 97 6.5.2 AR Coefficient Estimation for ARMA Signals As an application of the Capon method for covariance estimation, we include here an example on how to use the Capon covariance estimates to find the AR coefficients of ARMA signals via the overdetermined modified Yule-Walker (OMYW) method [14] [69] [75]. We first briefly explain the OMYW method. For an ARMA(p, q) process, the covariance sequence and the AR coefficients are related by r(q) r(q-l) r(q-p + 1) a i r(q + l) r{q + 1) r(q) r(q-p + 2) 2 = r{q + 2) r(q + L 1) r(q + L 2) r(q + L-p) _ dp _ r(q + L) _ (6.21) If L = p in (6.21), then we have a system of q equations with q unknowns. These equations are referred to as the modified Yule-Walker equations [14] since they con stitute a generalization of the Yule-Walker equations for the AR signals. Replacing the theoretical covariances (r(A:)} by their sample estimates {r(k)} in (6.21) yields f(q) r(q-l) 1 + h1 1 1 r(q + 1) r(q + 1) r(q) f(q-p + 2) 2 r{q + 2) r(q + L 1) f(q + L- 2) r(q + L p) _ dp _ r(q + L) _ (6.22) The overdetermined case of L > p in (6.22) is motivated by the fact that additional information in the higher-lag covariances can be exploited to improve the accuracy of the AR coefficient estimates; that is, we can make use of the additional information by choosing L > p and solving the so-obtained overdetermined system of equations, either in a least-squares (LS) or in a total-least-squares (TLS) sense [30]. Cl which shows that MAFI(L, K, K) = WLSE(L, 0, K). (5.63) The MAFI interpretation of the WLS method, afforded by the above analysis, is interesting. In particular, it makes a clear connection between using the MAFI and the WLS techniques for amplitude estimation. The MAFI approach is however more general than the WLS technique. As an example, we derive a new MAFI amplitude estimator that does not belong to the WLS class as follows. Let zk{l) and vk{l) denote the fc-th element of z(V) and, respectively, u(l) in (5.60). Then zk{l) = ake^1 + uk(l), k = 1,2,..., K. (5.64) The above equations are related to one another only via the correlation between vk{l) and up(l) (for k ^ p). If we ignore the correlation, then the MAFI(L, K, 1) estimate of the ak via LS is given by 1 L~l &k = ~J^zk(l)e-^1. (5.65) ^ 1=0 Unlike the Capon (5.44) and APES (5.45) estimators (which can also be shown to be members of the MAFI(L,iF, 1) class [70]), the above MAFI(L, K, 1) estimator does require the knowledge of the number and frequencies of the sinusoids, which makes it behave more like a MAFI(L, K, K) estimator. In particular, it performs quite well for cases where some sinusoids are closely spaced, as will be seen in Section 5.5. Other interesting MAFI amplitude estimators may be devised by using some other choices of H in lieu of the one given in (5.58), as the solution to (5.46) is not unique. Specifically, one may introduce certain unitary transform on the H in (5.58), or choose K < K, or replace the Ik in (5.59) by another nonsingular matrix, which all lead to solutions that are different from (5.58). Furthermore, one could even change the criterion in (5.46) to another reasonable definition of the generalized SNR. However, such variations on the theme of MAFI are beyond the scope of the present work. TABLE OF CONTENTS ACKNOWLEDGEMENTS iii LIST OF TABLES vii LIST OF FIGURES viii ABSTRACT xi CHAPTERS 1 INTRODUCTION 1 1.1 Background and Scope of the Work 1 1.1.1 Capon, APES, and MAFI Spectral Estimators 1 1.1.2 Efficient Implementation 4 1.1.3 Amplitude Estimation 5 1.1.4 Covariance Sequence Estimation 5 1.2 Significance and Contributions 6 1.3 Organization of the Dissertation 7 2 LITERATURE SURVEY 9 2.1 Filterbank Approaches and Capon Estimator 9 2.2 Covariance Estimation 13 3 MAFI APPROACH TO SPECTRAL ESTIMATION 16 3.1 Forward and Backward Data Vectors 16 3.2 MAFI Filters 19 3.2.1 Capon Filter 20 3.2.2 APES Filter 21 3.2.3 Another Matched Filter 22 3.3 Analyses of MAFI Approaches 24 3.3.1 Computational Complexity 24 3.3.2 Statistical Performance 25 3.4 2-D Extensions 27 3.5 Numerical Examples 31 3.5.1 1-D Complex Spectral Estimation 31 3.5.2 2-D Complex Spectral Estimation 33 3.6 Summary 36 IV 6.1 Power spectral density estimates by using Capon-1 and Capon-2. The plots are the averages of 100 independent realizations, (a) N = 256, M = 50; (b) N = 32, M = 10 86 6.2 Pole-zero diagrams for ARMA test cases, (a) ARMAl; (b) ARMA2; (c) ARM A3; (d) ARMA4 93 6.3 True power spectral densities, (a) ARMAl; (b) ARMA2; (c) ARMA3; (d) ARMA4 94 6.4 True covariance sequences, (a) ARMAl; (b) ARMA2; (c) ARMA3; (d) ARMA4 95 6.5 Covariance sequence estimation with N 256 and M = 50. The mean-squared errors (MSEs) of the covariance estimates, normalized with respect to r(0), are based on 100 independent realizations, (a) ARMAl; (b) ARMA2; (c) ARMA3; (d) ARMA4 96 6.6 The AR coefficient estimation of the ARMA signals via the overde termined modified Yule-Walker method with N = 256 and M = 32. The curves are the summations of the mean-squared errors (MSE) of all the AR coefficient estimates versus the numbers of included equa tions, which have been set as 4, 8, 16, 32, 64 and 128, respectively. The MSE curves are based on 100 independent realizations, (a) ARMAl; (b) ARMA2; (c) ARMA3; (d) ARMA4 99 6.7 10 superimposed realizations of the MA covariance sequence estimates with N = 64 and M = 6. (a) The standard method; (b) The Capon method 100 x 17 By borrowing the notations defined in (3.49)(3.51), the solution fbm to (A.4) is straightforward to obtain: 1 1 FBM 12 1 /[ikiir ln "2"- 1,2 1|-H- '2 M2IM2 + jIM2IM2M3 ii^h2+^ i^f^i 1.. ..91 u 12 1 1 ff |2 1 I, 11 a I H 12 inr k ^ - ihi i*! h 2 \ H I2 K ^3 (A.5) Next we evaluate the FBA estimator as given by (3.53). We first compute the E : -i = i A IMf w**y imi -2 (A.6) where A = IMt IKI|2 2 ||i/2||2 2 H^H2 pf 1/3I + 4. (A.7) Substituting (A.7) into (3.53) and performing some simple manipulations, we obtain FBA = [4*^2 2 ||i/3||2 {^1^2) + 2(i/fi/3)(i/fi/3)*] /[Ikill2 ll^ll2 ll^sll2 -2 llalli2 lk2||2 2 ||izi||2 ||i/3||2 + 4 ||i/i||2 H^ill2 \v2t/3\2 - ||l23||2 ||/f I/2|2 + 2 |l/f l/2|2 + (l/f l/2)*(i/f I/3)(l2^i23)* + (i/f */2)(vf V3)*{v%v3) \\v2\\2 Il/f 1/3|2 + 2 |i/f I/3|2] (A.8) Comparing (A.5) and (A.8) confirms that FBA and FBM coincide. Hence (3.43) follows. 65 (a) (b) Figure 5.2: Empirical MSEs and the CRB versus local SNR when N = 32, M = 8, and the observation noise is colored (an AR(1) process), (a) 0:3. (b) 1. 34 (a) (b) (c) (d) Figure 3.2: Empirical bias and variance of the 1-D MAFI estimators as SNRj varies when N = 64 and M = 15. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. 119 [49] H. Li, P. Stoica, and J. Li. Computationally efficient maximum likelihood esti mation of structured covariance matrices. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 2325-2328, Seat tle, May 1998. [50] H. Li, P. Stoica, and J. Li. Computationally efficient maximum likelihood estima tion of structured covariance matrices. IEEE Transactions on Signal Processing, 47(5), May 1999. [51] H. Li, P. Stoica, J. Li, and A. Jakobsson. On the performance analysis of forward- only and forward-backward matched-filterbank spectral estimators. Proceedings of 31st Annual Asilomar Conference on Signals, Systems, and Computers, pages 1210-1214, Pacific Grove, California, November 1997. [52] J. Li and P. Stoica. An adaptive filtering approach to spectral estimation and SAR imaging. IEEE Transactions on Signal Processing, 44(6): 1469-1484, June 1996. [53] Z.-S. Liu, H. Li, and J. Li. Efficient implementation of Capon and APES for spectral estimation. IEEE Transactions on Aerospace and Electronic Systems, 34(4):1314-1319, October 1998. [54] L. Ljung. System Identification: Theory for the User. Prentice-Hall, Englewood Cliffs, NJ, 1987. [55] S. L. Marple, Jr. Digital Spectral Analysis with Applications. Prentice-Hall, Englewood Cliffs, NJ, 1987. [56] C. T. Mullis and L. L. Scharf. Quadratic estimators of the power spectrum, in S. Haykin, editor, Advances in Spectrum Analysis and Array Processing, Englewood Cliffs, NJ: Prentice-Hall, 1991. [57] D. C. Munson, Jr., J. D. OBrian, and W. K. Jenkins. A tomographic formulation of spot-light mode synthetic aperture radar. Proceedings of the IEEE, 71(8):917- 925, August 1983. [58] B. Musicus. Fast MLM power spectrum estimation from uniformly spaced cor relations. IEEE Transactions on Acoustics, Speech Signal Processing, 33:1333- 1334, October 1985. [59] R. Onn and A. O. Steinhardt. Multi-windows spectrum estimation a linear algebraic approach. International Journal on Adaptive Control and Signal Pro cessing, 7:103-116, 1993. [60] B. Porat. Some asymptotic properties of the sample covariance of Gaussian autoregressive moving-average processes. Journal of Time Series Analysis, 8(2):205-220, 1987. [61] K. Riedel and A. Sidorenko. Minimum bias multiple taper spectral estimation. IEEE Transactions on Signal Processing, 43(1): 188195, January 1995. [62] E. A. Robinson. A historical perspective of spectrum estimation. Proceedings of the IEEE, 70(9):885-907, September 1982. [63] R. Roy and Thomas Kailath. ESPRIT-estimation of signal parameters via ro tational invariance techniques. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(7):984-995, July 1989. 13 in practise. GSC also found applications in direct-sequence Code-Division Multiple- Access (CDMA) communication systems for blind multiuser detection [36] [66] [65]. An important extension of the LCML beamformer was made in [7] [6] for applica tions in airborne radar systems. A typical signal environment faced by such systems consists of strong clutter/interference of complicated angle-Doppler spectrum which is unknown and may be varying in both time and space. The extension made by Brennan el al is to simultaneously combine the signals received on multiple elements of an antenna array (the spatial domain) and from multiple pulse repetition peri ods (the temporal domain). Such a technique is referred to as space-time adaptive processing that has received much research interest recently [80] [82]. Another interesting application of the Capon spectral estimator is SAR imag ing [57] [3]. Conventional SAR imaging techniques are the FFT (Fast Fourier Trans form) or windowed-FFT methods. A number of parametric spectral estimation meth ods have also been used for SAR imaging [26] [38] [33], though their interest in SAR imaging is limited because of their sensitivity to model errors. A comparative study in [24] showed that adaptive filterbank approaches such as Capon offers good SAR images and enjoys the advantage of robustness as compared to parametric methods. A number of modified Capon methods have also been suggested for SAR imaging [25] [4], 2.2 Covariance Estimation One type of the covariance estimation problems is to estimate the covariance sequence from a finite number of data samples. A standard technique for estimating covariance sequences uses the biased or unbiased sample covariance estimator. The biased covariance estimator is more commonly used because it provides smaller MSEs than the unbiased one and guarantees the covariance estimates to be non-negative [55]. The problem of the sample covariance estimators is the unrealistic windowing 28 where L\ = Ni Mi 4- 1 and L2 N2 M2 + 1. Let y(/i,/2) = vec[Y (h,l2)\, (3.61) y(h,l2) = vec[Y(/i, 2)], (3.62) where vec[-] denotes the operation of stacking the columns of a matrix on top of each other. Let aM!,M2(wi>^2) 3-m2(w2) aMl(u}i), where denotes the Kronecker matrix product, and aMjfe(^fc) 1 ejk T ei{Mk-\)uk k = 1,2. (3.63) (3.64) Then y(Zi, l2) and y(h, h) can be written as y(b, h) = [tt(wi,w2)aMllM2(w1,w2)]d(ll+22) + ea,li(J2(Zi, l2), (3.65) y(h,l 2) = [a{uJi,u2)aMuM2{vi,u2)]eullll+J2l2) + eUuul2(h,l2), (3.66) where a(u)i,u2) = a*(Â£Ji,a;2)e-J^7Vl-1^Wle-J^JV2-1^W2, (3.67) and WliW2(Zi, Z2) and eWliW2(Zi, Z2) are, respectively, formed from {eWli,2(ni, n2)} in the same ways as y(Zi,Z2) and y{li,l2) are made from {y(ni,n2)}. Suppose that the initial phase of the sinusoidal signal of (3.59) is a random variable uniformly distributed over the interval [0, 27t) and independent of the noise term. Then the covariance matrix of y(l\,l2) or, equivalently, of y(Zi,Z2) is given by R= \a(u}Uuj2)\2aMuM2{^\,^2)a^lM2{u)i,u>2) + Q(uq,u;2), (3.68) where Q(cji,cu2) is the covariance matrix of eu)li,2(Z1,Z2) or eWliW2(Zi, Z2). By making use of the fact that y(Zi, h) Jy*(Ti Zi 1, L2 l2 1), (3.69) 60 where T denotes some nonsingular matrix that makes X semi-unitary. One such T IS T = (AH CT1 A)~1/2. (5.54) Hence, X = Q-1/2A(AhQ-1A)-1/2. (5.55) We next observe that H = Q-1/2X (5.56) satisfies (5.47). Consequently, we have H = Q~1A(AhQ~1A)~1/2. (5.57) The final step is to observe that post-multiplying H by a nonsingular matrix does not change the generalized SNR criterion. Then, it follows immediately that H = Q^AA^Q-^A)-1 (5.58) maximizes the generalized SNR and it also satisfies the constraint H^A = I*. (5.59) The constraint (5.59) says that each (row) filter in HH passes one sinusoid undis torted, and completely annihilates the others. From (5.27) and (5.28), the filtered data corresponding to (5.58) is given by z(0 = HHy(/) = D*a + HHe(Z) = Da + i/(Z), / = 0,1,..., L 1. (5.60) The covariance matrix of i/(/) can be estimated as H"QH = (AhQ-1A)_1. (5.61) It follows that the WLS (Markov-like) estimate of a in (5.60) is given by a = L1 D//i(AifQ1 A)D; 11=0 L-1 -1 r L1 X]Df(A/Q-1A)(A/Q-1A)-1AiQ-1y(0 1=0 XAfQ-'A, (=0 L-1 ^AfQ-'y (i) 11=0 (5.62) 50 5.2 LS Amplitude Estimators We consider two LS methods in this section, namely LSE(1,0, K) and LSE(1, 0,1). 5.2.1 LSEfl, 0. K) This is perhaps the most direct approach. Let us write the available data sequence in the following form x(0) 1 1 1 1 o' p *(1) ejwi ejUK 2 + n(l) I tH 1 ej(N- 1)cli g j(N-l)uiK aK 1 CS 1 h-1 1 or, with obvious definitions, x = a + v, (5.3) which is a linear regression equation. The LS estimate of a is = (^^^x, (5.4) where (-)H denotes the conjugate transpose. Note that the noise is not modeled, even though it may be correlated. Despite this fact, LSE(1,0, K) is asymptotically efficient [31]. A relatively simple manner to see this is as follows. First, note that E{ct} a, (5.5) where E{-} denotes the statistical expectation. The Mean Squared Error (MSE) of L S MSE{q:} covjo:} = E {(o; a) (a oi)H} = (")-1/W()-1, (5.6) where w = (5.7) 69 10 1 10 o Â£10' LJJ ~o CD lio'2 cr C/D I c C0 -3 0>10 10" 10" N=32,o2=1 O-2,f3=0.3,Colored Noise T a Capon 1 : X- X APES1 _ 0 0 CaponK : O- o APESK :i + + MAFI1 ; CRB .....0" : .9 J I I I I L 0 2 4 6 8 10 12 14 16 Subvector Length M (a) N=32,a2=10-2,f1=0.1 .Colored Noise = I i 1 i 1 I i it run a Caponl -= : X- X APES1 : 1. 0. 4 CaponK j- ! G o APESK : - O + + MAFI1 / - _ CRB .' = : r -x.., D J - +. x- : o. 4 ,9 0 o' : e o... +'' : 9 + : r 1 i i i L I i i 1 0 2 4 6 8 10 12 14 16 Subvector Length M (b) Figure 5.4: Empirical MSEs and the CRB versus M when N = 32 and the observation noise is colored (an AR,(1) process with a2 = 0.001). (a) a3. (b) a\. 62 5.5 Numerical Examples In what follows, we investigate the performances of the various amplitude estimators described in the previous sections. For notational simplicity, we will refer to these methods as follows: LSE1: LSE(1,0,1) using (5.14); LSEK: LSE(1,0,K) using (5.4); Caponl: WLSE(L,0,1) using (5.44); APES1: WLSE(L, 0,1) using (5.45); CaponK: WLSE(L,0,/F) using (5.38); APESK: WLSE(L,0, K) using (5.29) along with (5.43); MAFI1: MAFI(L, K, 1) using (5.65) along with (5.43). We will compare these methods with one another as well as the CRB given in (5.11). Since all these methods are asymptotically efficient, we only consider the case when N is relatively small. Specifically, we choose N = 32. The data consist of three complex sinusoids corrupted by a complex Gaussian noise v(n) (to be specified): x(n) = s(n) + v(n), n 0,1,..., N 1, (5.66) where 3 s(n) = '}Takej2*h. (5.67) fc=i The frequencies of the sinusoids are /i = 0.1, /2 = 0.11, and = 0.3. Also, ai = e-771'/4, o2 = eJir/3, and 0:3 = eJ7r/4. All examples are based on 200 Monte-Carlo simulations. The MSE figures shown in what follows are obtained as 1 200 MSE{cbJ = goo ~ ak u =1 (5.68) 48 are of interest to us) are close to one another. Statistical analyses that compare the merits of the two LS methods are also provided in Section 5.2. Since the LS methods completely ignore the correlation of the observation noise, they are in general suboptimal. By splitting the data vector into a number of subvectors, the covariance matrix of the noise-only part of the data subvectors can be estimated, which makes it possible to use a Markov-like estimator that is optimal in the class of Weighted Least Squares (WLS) techniques [68]. We describe in Section 5.3 several ways for estimating the aforementioned covariance matrix, which lead to different WLS amplitude estimators. Additionally, we show that, if the restriction of estimating one amplitude at a time is again imposed, we obtain two WLS amplitude estimators that are equivalent to the Capon [16] [44] and APES [52] methods extensively used for spectral analysis. The observation that some general spectral estimators, such as Capon and APES, can be used to solve the problem posed in (5.1) motivated us to seek other relatively sophisticated spectral analysis techniques for amplitude estimation. Both Capon and APES belong to the general class of interbank approaches to spectral estimation [75], which involve splitting the data into subvectors, passing them through a set of narrowband filters (filterbank) whose center frequencies correspond to those that are of interest to us, and, finally, estimating the spectral density function at those frequencies from the filtered and, hopefully, signal-enhanced data. As one can see, the key issue of filterbank approaches is the design of the filters. A recent study has suggested the choice of matched filters, which gave rise to the MAtched-FIlterbank (MAFI) approach to spectral estimation [70]. Even though neither Capon nor APES was derived in the MAFI framework (see [16] and [52] for their original derivations), it was found that both are members of the MAFI approach [70]. In the light of the work of [70], we derive in Section 5.4 a generalized MAFI approach to amplitude estimation. Interestingly enough, we show that, under certain circumstances, MAFI 120 [64] R. O. Schmidt. Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas and Propagation, 34(3):276280, March 1986. [65] J. B. Schodorf and D. B. Williams. Array processing techniques for multiuser detection. IEEE Transactions on Communications, 45(11): 13751378, November 1997. [66] J. B. Schodorf and D. B. Williams. A constrained optimization approach to multiuser detection. IEEE Transactions on Signal Processing, 45(1) :258262, January 1997. [67] A. Schuster. The periodogram and its optical analogy. Proceedings of the Royal Society of London, 77, Series AT36-140, June 1906. [68] T. Soderstrom and P. Stoica. System Identification. Prentice-Hall, London, 1989. [69] P. Stoica, B. Friedlander, and T. Soderstrom. Approximate maximum-likelihood approach to ARMA spectral estimation. International Journal of Control, 45(4): 12811310, 1987. [70] P. Stoica, A. Jakobsson, and J. Li. Matched-filter bank interpretation of some spectral estimators. Signal Processing, 66(1):4559, April 1998. [71] P. Stoica, H. Li, and J. Li. Amplitude estimation of sinusoidal signals: Survey, new results, and an application. To appear in IEEE Transactions on Signal Processing. [72] P. Stoica, H. Li, and J. Li. A new derivation of the APES filter. To appear in IEEE Signal Processing Letters, 1999. [73] P. Stoica, H. Li, and J. Li. Spectral estimation and radar imaging. Proceedings of 30th ISCIE International Symposium on Stochastic Systems Theory and Its Applications, Kyoto, Japan, November 1998. [74] P. Stoica, H. Li, and J. Li. Amplitude estimation with applications to system identification. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 1777-1780, Phoenix, Arizona, March 1999. [75] P. Stoica and R. L. Moses. Introduction to Spectral Analysis. Prentice-Hall, Upper Saddle River, NJ, 1997. [76] P. Stoica and T. Soderstrom. On reparameterization of loss functions used in estimation and the invariance principle. Signal Processing, 17:383-387, August 1989. [77] D. J. Thomson. Spectrum estimation and harmonic analysis. Proceedings of the IEEE, 70(9):1055-1096, September 1982. [78] T. J. Ulrych and T. N. Bishop. Maximum entropy spectral analysis and autore gressive decomposition. Reviews of Geophysics and Space Physics, 13:183-200, February 1975. [79] B. D. Van Veen and K. M. Buckley. Beamforming: A versatile approach to spatial filtering. IEEE Signal Processing Magazine, pages 4-24, April 1988. 101 the approximate Capon method, which avoids the spectral factorization and makes use of FFT, may be preferred in practical application. 20 1 L_1 g(w) = ZY1 y(l)e~JUl- (3-24) ^ =o Since Q(a>) is Toeplitz and, therefore, persymmetric, one can show that the hw in (3.17) satisfies [52] - hwe-j(M-1)w. (3.25) Consequently, it follows after some calculation that (3.22) is equivalent to &FB(u) = h*g(u). (3.26) Hence, due to the persymmetry of Q(u>), the FB estimate of a{u) has the same form as the FO estimate of a(u>) (see (3.21) (3.26)) However, the filter vector hw obtained, with the FO approach is in general different from that corresponding to the FB approach [52]. Although neither Capon nor APES was derived in the MAFI framework (for original derivations of these methods, we refer to [16] [75] [52]), in what follows we show that two natural estimators of Q(w) in (3.17) lead to the Capon and APES filters, respectively. More interestingly, we also show that even though a third natural estimator of Q(cj) gives a new filter which is different from the former two, the spectral estimator corresponding to the new filter turns out to be equivalent to APES as well. 3.2.1 Capon Filter By (3.10), one natural choice is to estimate Q(u;) as QCaponM = R ~ |(w)|2 aM(w)aÂ£( turn, leads to FOC or FBC, respectively. (For notational simplicity, in what follows the forward-only Capon, APES, and MAFI spectral estimators will be referred to as FOC, FOA, and FOM, respectively. Likewise, FBC, FBA, and FBM represent the corresponding forward-backward counterpart estimators.) By making use of the APPENDIX D PROOF OF THEOREM 6.4.1 Lemma D.0.1 MlM-1 M-1M-1 M-lM-j-1 (D.l) Proof: M-1W-1 Ml Ml f(k>k j)= k k=0 j=0 j=0 fc=0 Ml Ml Ml j1 = + (D-2) j0 kj j= 1 fc=0 Let s = k j. Note that the first term of the right side of (D.2) corresponds to s > 0 and the second term corresponds to s < 0. Since 0 < k < M 1 and 0 < j < M 1, it is obvious that M 1 > k = s + j > s, for s > 0; 0 < k = s + j < s + M 1 = M |s| 1, for s < 0. Therefore, we can rewrite (D.2) as Ml Ml -1 M-\s\-l 'Â¡>2J2f{k,s)+ Â£ /(m) s=0 k=s s=(M1) k=0 Ml Ml Ml Msl = ^2^2f(k^s) + f(k~s)> s=0 k=s s=l fc=0 which concludes the proof of Lemma D.0.1. (D.3) (D.4) 114 MATCHED-FILTERBANK APPROACHES TO SPECTRAL ANALYSIS AND PARAMETER ESTIMATION By HONGBIN LI A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY. UNIVERSITY OF FLORIDA 1999 J 35 1-D Complex Amplitude Estimation 1-D Complex Amplitude Estimation 1-D Complex Amplitude Estimation 1-D Complex Amplitude Estimation (C) (d) Figure 3.3: Empirical bias and variance of the 1-D MAFI estimators as the filter length, M, varies when N = 64 and SNRi = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. 67 (b) Figure 5.3: Empirical MSEs and the CRB versus local SNR when N = 32, M = 8, and the observation noise is white, (a) 0:3. (b) a\. 83 The estimator given in (6.1) is consistent if the ergodicity assumption is satis fied. A study on whether the standard sample covariance estimator is also asymptot ically statistically efficient, i.e., whether it asymptotically achieves the Cramr-Rao Bound (CRB), has been undertaken in [60]. Let y(t) be an ARMA(p,q) signal. If p > q, then the sample covariance estimate r(k) is asymptotically statistically effi cient if and only if 0 < k < p q\ in particular, for AR processes of order p, r(k) is asymptotically efficient for 0 < k < p, but inefficient for all other k. If p < q, none of r(k) is asymptotically efficient; in particular, none of r(k) is asymptotically efficient for an MA process. It is known that the sample covariance sequence (r(0),..., r(N 1), 0, 0,...} and the data periodogram constitute a Fourier transform pair. It is also known that the periodogram is a statistically inefficient (in fact inconsistent) estimator of the PSD [75]. This observation suggests that better covariance estimators might be obtained by Fourier inverting better PSD estimators. However, this is not necessarily so. Briefly stated, the reason is that the Fourier transform and the inverse Fourier transform are integral transforms and hence small errors in one domain may be associated with large errors in the other domain or vice versa. The fact that the covariance estimates in (6.1) are consistent whereas the periodogram is not illustrates this observation. Also note that, while the Burg estimate of the PSD is typically more accurate than the periodogram, the corresponding Burg estimate of the covariance sequence is generally poorer than (6.1) [78]. In spite of the fact briefly discussed above, in the following we consider es timating the covariance sequences by inverting a PSD estimate that is often much more accurate than the periodogram, namely the Capon PSD estimate. Like the pe riodogram, no model is assumed in the Capon PSD estimator, which makes it more robust than the parametric estimators in many situations. Although it has lower spectral resolution than the AR spectral estimator, it generally exhibits less variance 78 a-Parameters b-Parameters Computational Complexity v... V (85) with LSEK ... ... (85) with APES1 o (85) with MAFI1 0- o (82) with APES1 OEM Q... a PEM Data Length N (c) Figure 5.7: Averaged RMSEs and the number of flops versus N for the second system when the observation noise is colored (an AR(1) process with a2 = 0.01) and M = 20 for APES1 and MAFI1. (a) RMSE of a-parameters. (b) RMSE of 6-parameters, (c) Number of flops. of flops, as N increases. As one can see, the initial system parameter estimates given by APES1 or MAFI1 are significantly better than those given by OEM, and yet the former two are computationally more efficient that the latter. 5.7 Summary This chapter examines the problem of amplitude estimation of sinusoidal sig nals in colored noise. Three general classes of estimators, namely the LS, WLS, and MAFI approaches to amplitude estimation, are discussed. It is shown that, under 15 Extended Invariance Principle (EXIP) [76] was recently presented in [49] [50]. This method provides asymptotic (for large samples) ML estimation for structured covari ance matrices. A closed-form solution for the estimation Hermitian Toeplitz matrices is obtained which makes the proposed method computationally much simpler than most existing Hermitian Toeplitz matrix estimation algorithms. Additionally, it was also shown that using the technique in such array processing algorithms as MUSIC [64] and ESPRIT [63] makes them achieve the Cramr-Rao Bound (CRB) for angle estimation, i.e., the best performance for any unbiased methods. 117 [15] J. A. Cadzow. Signal enhancement a composite property mapping algorithm. IEEE Transactions on Acoustics, Speech, and Signal Processing, 36(1):4962, January 1988. [16] J. Capon. High resolution frequency-wavenumber spectrum analysis. Proceedings of the IEEE, 57(8): 14081418, August 1969. [17] J. Capon. Correction to probability distributions for estimators of the frequency- wavenumber spectrum. Proceedings of the IEEE, 59(1): 112, January 1971. [18] J. Capon. Maximum-likelihood spectral estimation, in S. Haykin, editor, Non linear Methods of Spectral Analysis, New York: Springer-Verlag, 1983. [19] J. Capon and N. R. Goodman. Probability distributions for estimators of the frequency-wavenumber spectrum. Proceedings of the IEEE, 58(10):1785-1786, October 1970. [20] D. G. Childers, editor. Modern Spectrum Analysis. IEEE Press, New York, 1978. [21] H. Cox. Resolving power and sensitivity to mismatch of optimum array proces sors. J. Acoust. Soc. Am., 54:771-785, September 1973. [22] P. J. Daniell. On the theoretical specification and sampling properties of au- tocorrelated time-series. Journal of Royal Statistical Society, 8, Series B:88-90, 1946. [23] S. Degerine. Maximum likelihood estimation of autocovariance matrices from replicated short time series. Journal of Time Series Analysis, 8(2):135-146, 1987. [24] S. R. DeGraaf. SAR imaging via 2-D spectral estimation methods. SPIE Pro ceedings on Optical Engineering in Aerospace Sensing, 2230, Orlando, FL, April 1994. [25] S. R. DeGraaf. Sidelobe reduction via adaptive FIR filtering in SAR imagery. IEEE Transactions on Image Processing, 3(3):292-301, May 1994. [26] S. R. DeGraaf. Parametric estimation of complex 2-d sinusoids. IEEE Fourth Annual ASSP Workshop on Spectrum Estimation and Modeling, pages 391-396, August, 1988. [27] O L. Frost, III. An algorithm for linearly constrained adaptive array processing. Proceedings of the IEEE, 60(8):926-935, August 1972. [28] D. R. Fuhrmann. Application of Toeplitz covariance estimation to adap tive beamforming and detection. IEE Transactions on Signal Processing, 39(10):2194-2198, October 1991. [29] D. R. Fuhrmann and M. I. Miller. On the existence of positive-definite maximum- likelihood estimates of structured covariance matrices. IEEE Transactions on Information Theory, 34(4):722-729, July 1988. [30] G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins Uni versity Press, Baltimore, MD, 1996. [31] U. Grenander and M. Rosenblatt. Statistical Analysis of Stationary Time Series. Almqvist och Wiksell, Stockholm, 1956. 64 most SNRs. The reason is that the approximation made in (5.37) is valid only for large N and M, which is not the case in this example. Figure 5.2(a) also shows that both LSE1 and Caponl are inconsistent (in SNR). Their inconsistency is not sur prising because both are biased estimators. Recall that the bias of LSE1, as given in (5.17), does not vanish unless N goes to infinity. Similarly, Caponl is always biased (downward) for finite N [70] [47], Figure 5.2(b) shows the counterpart curves for aq. (The results for a2 are omitted because they resemble those for oq.) Note that /2 /i = 0.01, which is smaller than l/N 0.03, the Fourier resolution limit. The performance degrades for all estimators under study, especially for LSE1, Caponl, and APES1, which estimate only one amplitude at a time. As shown in Figure 5.2(b), LSE1 and Caponl essentially fail for all SNRs considered due to their large MSEs. APES1 is no longer close to the CRB but, unlike the previous two estimators, it still appears to be consistent (in SNR). As in Figure 5.2(a), CaponK again deviates away from the CRB at high SNRs. It appears that the approximation made in (5.37) introduces a bias (at small N and M) that may be negligible at low SNRs but dominates the variance at high SNRs. The bias does not disappear as the SNR increases, which causes the divergence of CaponK from the CRB. APESK performs quite well for high SNRs; however, it is not very stable at low SNRs (due to large variance). The best estimator in this example is MAFI1. The knowledge of the number and locations of the sinusoids, which the other one-at-a-time estimators may spare but is indispensable to MAFI1, appears to play an important role in its good performance in the current case that fails the other one-at-a-time estimators. As stated in Section 5.2, LSEK is statistically efficient, i.e. it achieves the CRB for any N > K, when the observation noise is white. To see how the other suboptimal (in finite samples) methods perform in such a case, we consider an example which is similar to the previous one except that v(n) is replaced by a zero-mean complex white 58 (5.45) are recognized to have the same form as the Capon [16] [44] and, respectively, the APES [52] spectral estimators. The two estimators were derived in [70] [47] by a different approach, namely the MAFI approach, which we will consider in a generalized form in the next section. It is interesting that the above two amplitude estimators, while both asymp totically efficient (and hence equivalent), have quite different finite-sample properties. Specifically, it was shown in [70] [47] that (5.44) is biased downward, whereas (5.45) is unbiased (within a second-order approximation) and in general has a better per formance than the former. 5.4 MAFI Amplitude Estimators In this section, we derive a generalized MAFI approach to amplitude estima tion. Let 6 CKxM be a matrix each row of which is a Finite Impulse Response (FIR) filter (for some 1 < K < M yet to be specified). The MAFI idea can be explained as follows: a) Design so that, when applied to {y(Z)}, it maximizes the SNR at the K filter outputs. b) Estimate the amplitudes from the filtered data (whose SNR should be higher than that in the raw data) by, e.g., the LS or WLS technique. Mathematically, H can be obtained as follows: H = argmaxtr (HhQH)_1H/(APA/)H H (5.46) Generalized SNR where H is constrained in a way that is specified later (in particular, to guarantee that H is finite), and tr(-) denotes the trace of a matrix. Let X" = (H"QH)-1/2H''Q1/2, (5.47) 49 amplitude estimators have equivalent forms to the WLS methods. However, the MAFI approach is more general than the WLS technique in that the latter is a special case of the former. To show this, a new MAFI amplitude estimator that does not fall into the WLS category is described in Section 5.4. Other interesting MAFI amplitude estimators may exist and are yet to be discovered. A common feature of the amplitude estimators considered in this chapter is that none of them models the observation noise exactly. Even so, all methods are asymptotically statistically efficient, that is, they all achieve the Cramr-Rao Bound (CRB) in large samples. However, their finite-sample properties, which are of primary interest to this work, are quite different. Since the finite-sample analysis is intractable in most cases, we use Monte-Carlo simulations in Section 5.5 to compare these methods with one another. The amplitude estimation problem in (5.1) occurs in a variety of signal pro cessing applications (see, e.g., [41] [55], and the references therein). In Section 5.6, we discuss its application to system identification. We show that, by using appropri ate amplitude estimators, we can avoid the iterative search required by the standard system identification routines, such as the Output Error Method (OEM) [68], and achieve very good performance at a usually reduced computational load. In concluding this section, we introduce the following notation to distinguish among the various amplitude estimators. For instance, LSE(1,0,1) denotes the LS estimator that does not split the data (and hence it uses one data snapshot), uses no prefiltering, and estimates one amplitude at a time. Likewise, MAFI(L, K, K) denotes the MAFI estimator that splits the data into L subvectors, utilizes a bank of K prefilters, and estimates K amplitudes simultaneously. The remaining amplitude estimators are similarly designated. PAGE 1 0$7&+('),/7(5%$1. $3352$&+(6 72 63(&75$/ $1$/<6,6 $1' 3$5$0(7(5 (67,0$7,21 %\ +21*%,1 /, $ ',66(57$7,21 35(6(17(' 72 7+( *5$'8$7( 6&+22/ 2) 7+( 81,9(56,7< 2) )/25,'$ ,1 3$57,$/ )8/),//0(17 2) 7+( 5(48,5(0(176 )25 7+( '(*5(( 2) '2&725 2) 3+,/2623+< 81,9(56,7< 2) )/25,'$ - PAGE 2 7KLV ZRUN LV GHGLFDWHG WR P\ ZLIH +RQJ PAGE 3 $&.12:/('*(0(176 ZRXOG OLNH WR H[SUHVV P\ VLQFHUH JUDWLWXGH WR P\ DGYLVRU 'U -LDQ /L IRU KHU FRQVWDQW VXSSRUW HQFRXUDJHPHQW HQWKXVLDVP DQG SDWLHQFH LQ JXLGLQJ WKLV UHVHDUFK 0\ GHHSHVW DSSUHFLDWLRQ JRHV WR 'U 3HWUH 6WRLFD IRU KLV QXPHURXV LQVLJKWIXO UHPDUNV DQG VXJJHVWLRQV ZKLFK FRPSUHKHQVLYHO\ LQIOXHQFHG WKLV ZRUN 6SHFLDO WKDQNV DUH GXH WR 'UV -RVH & 3ULQFLSH )UHG 7D\ORU :LOOLDP : (GPRQVRQ DQG 'DYLG & :LOVRQ IRU VHUYLQJ RQ P\ VXSHUYLVRU\ FRPPLWWHH DQG IRU WKHLU FRQWULEXWLRQ WR P\ JUDGXDWH HGXFDWLRQ DW WKH 8QLYHUVLW\ RI )ORULGD DOVR ZLVK WR WKDQN =KDRTLDQJ %L 5REHUW 6WDQILOO DQG RWKHU IHOORZ JUDGXDWH VWXGHQWV ZLWK ZKRP KDG WKH JUHDW SOHDVXUH RI LQWHUDFWLQJ 'UV =KHQJ6KH /LX DQG *XRTLQJ /LX KDYH P\ JUDWLWXGH IRU VKDULQJ PDQ\ LQWHUHVWLQJ GLVFXVVLRQV ZLWK PH ZRXOG OLNH WR JUDWHIXOO\ DFNQRZOHGJH DOO WKH SHRSOH ZKR KHOSHG PH GXULQJ P\ 3K' SURJUDP LLL PAGE 4 7$%/( 2) &217(176 $&.12:/('*(0(176 LLL /,67 2) 7$%/(6 YLL /,67 2) ),*85(6 YLLL $%675$&7 [L &+$37(56 ,1752'8&7,21 %DFNJURXQG DQG 6FRSH RI WKH :RUN &DSRQ $3(6 DQG 0$), 6SHFWUDO (VWLPDWRUV (IILFLHQW ,PSOHPHQWDWLRQ $PSOLWXGH (VWLPDWLRQ &RYDULDQFH 6HTXHQFH (VWLPDWLRQ 6LJQLILFDQFH DQG &RQWULEXWLRQV 2UJDQL]DWLRQ RI WKH 'LVVHUWDWLRQ /,7(5$785( 6859(< )LOWHUEDQN $SSURDFKHV DQG &DSRQ (VWLPDWRU &RYDULDQFH (VWLPDWLRQ 0$), $3352$&+ 72 63(&75$/ (67,0$7,21 )RUZDUG DQG %DFNZDUG 'DWD 9HFWRUV 0$), )LOWHUV &DSRQ )LOWHU $3(6 )LOWHU $QRWKHU 0DWFKHG )LOWHU $QDO\VHV RI 0$), $SSURDFKHV &RPSXWDWLRQDO &RPSOH[LW\ 6WDWLVWLFDO 3HUIRUPDQFH ([WHQVLRQV 1XPHULFDO ([DPSOHV &RPSOH[ 6SHFWUDO (VWLPDWLRQ &RPSOH[ 6SHFWUDO (VWLPDWLRQ 6XPPDU\ ,9 PAGE 5 ()),&,(17 ,03/(0(17$7,21 2) &$321 $1' $3(6 ,QWURGXFWLRQ (IILFLHQW ,PSOHPHQWDWLRQ RI $3(6 ([WHQVLRQ WR &DSRQ 1XPHULFDO DQG ([SHULPHQWDO ([DPSOHV 6XPPDU\ $03/,78'( (67,0$7,21 ,QWURGXFWLRQ /6 $PSOLWXGH (VWLPDWRUV /6( .f /6(f :/6 $PSOLWXGH (VWLPDWRUV :/6(Â/ .f :/6(/ f 0$), $PSOLWXGH (VWLPDWRUV 1XPHULFDO ([DPSOHV (VWLPDWLRQ 3HUIRUPDQFH YHUVXV 615 7KH (IIHFW RI 0 $SSOLFDWLRQ WR 6\VWHP ,GHQWLILFDWLRQ 6\VWHP ,GHQWLILFDWLRQ 8VLQJ $PSOLWXGH (VWLPDWLRQ 1XPHULFDO ([DPSOHV 6XPPDU\ &$321 (67,0$7,21 2) &29$5,$1&( 6(48(1&(6 ,QWURGXFWLRQ 6WDQGDUG &RYDULDQFH (VWLPDWRU DQG 2XWORRN &DSRQ 36' (VWLPDWRU &DSRQ &RYDULDQFH (VWLPDWRU ([DFW 0HWKRG $SSUR[LPDWH 0HWKRG &RPSXWDWLRQDO $VSHFWV 1XPHULFDO 5HVXOWV $50$ &RYDULDQFH (VWLPDWLRQ $5 &RHIILFLHQW (VWLPDWLRQ IRU $50$ 6LJQDOV 0$ 0RGHO 2UGHU 'HWHUPLQDWLRQ 6XPPDU\ &21&/86,216 6XPPDU\ 5HPDUNV )XWXUH :RUN $33(1',;(6 $ 3522) 2) f $1' f % 3522) 2) 7+(25(0 Y PAGE 6 & 3522) 2) 7+(25(0 3522) 2) 7+(25(0 5()(5(1&(6 %,2*5$3+,&$/ 6.(7&+ YL PAGE 7 /,67 2) 7$%/(6 &KRLFH RI 0 IRU WKH :/6 DQG 0$), DPSOLWXGH HVWLPDWRUV &RPSDULVRQ RI WKH FRPSXWDWLRQDO EXUGHQV RI WKH VWDQGDUG DQG &DSRQ PHWKRGV ZLWK 1 DQG Â“ fÂ§ 7KH $50$ SURFHVVHV XVHG LQ WKH QXPHULFDO VLPXODWLRQV YLL PAGE 8 /,67 2) ),*85(6 6\QWKHWLF $SHUWXUH 5DGDU 6$5f LPDJHV RI D VLPXODWHG 0,* DLUn SODQH REWDLQHG E\ XVLQJ Df ))7 Ef ZLQGRZHG ))7 Ff &DSRQ DQG Gf $3(6 7KH FRPSOH[ DPSOLWXGH RI WKH VXP RI VLQXVRLGV XVHG LQ WKH VLPXODWLRQV Df 5HDO SDUW Ef ,PDJLQDU\ SDUW (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV 615L YDULHV ZKHQ ,9 DQG 0 Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLn PDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV WKH ILOWHU OHQJWK 0 YDULHV ZKHQ 1 DQG 615L G% Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV WKH 615L YDULHV ZKHQ 1[ 1 DQG 0[ 0 Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH PAGE 9 (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV WKH ILOWHU OHQJWK 0 0L 0 YDULHV ZKHQ 1L 79 DQG 615[ G% Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH 6$5 LPDJHV REWDLQHG IURP WKH (5,0 GDWD E\ XVLQJ Df ))7 Ef ZLQGRZHG ))7 Ff &DSRQ DQG Gf $3(6 36' RI WKH WHVW GDWD WKDW FRQVLVW RI WKUHH VLQXVRLGV DQG DQ $5f QRLVH SURFHVV (PSLULFDO 06(V DQG WKH &5% YHUVXV ORFDO 615 ZKHQ 79 0 DQG WKH REVHUYDWLRQ QRLVH LV FRORUHG DQ $5f SURFHVVf Df D Ef D[ (PSLULFDO 06(V DQG WKH &5% YHUVXV ORFDO 615 ZKHQ 79 0 DQG WKH REVHUYDWLRQ QRLVH LV ZKLWH Df D Ef D[ (PSLULFDO 06(V DQG WKH &5% YHUVXV 0 ZKHQ 79 DQG WKH REVHUn YDWLRQ QRLVH LV FRORUHG DQ $5f SURFHVV ZLWK R f Df R Ef D[ $YHUDJHG 506(V DQG WKH QXPEHU RI IORSV YHUVXV 79 IRU WKH ILUVW V\VWHP ZKHQ WKH REVHUYDWLRQ QRLVH LV ZKLWH R f DQG 0 fÂ§ IRU $3(6 DQG 0$), Df 506( RI DSDUDPHWHUV Ef 506( RI SDUDPHWHUV Ff 1XPEHU RI IORSV 36' HVWLPDWH RI WKH RXWSXW RI WKH ILUVW V\VWHP FRUUXSWHG E\ ZKLWH QRLVH ZLWK D DQG 79 $YHUDJHG 506(V DQG WKH QXPEHU RI IORSV YHUVXV 79 IRU WKH VHFRQG V\VWHP ZKHQ WKH REVHUYDWLRQ QRLVH LV FRORUHG DQ $5f SURFHVV ZLWK D f DQG 0 IRU $3(6O DQG 0$), Df 506( RI D SDUDPHWHUV Ef 506( RI SDUDPHWHUV Ff 1XPEHU RI IORSV L[ PAGE 10 3RZHU VSHFWUDO GHQVLW\ HVWLPDWHV E\ XVLQJ &DSRQ DQG &DSRQ 7KH SORWV DUH WKH DYHUDJHV RI LQGHSHQGHQW UHDOL]DWLRQV Df 1 0 Ef 1 0 3ROH]HUR GLDJUDPV IRU $50$ WHVW FDVHV Df $50$O Ef $50$ Ff $50 $ Gf $50$ 7UXH SRZHU VSHFWUDO GHQVLWLHV Df $50$O Ef $50$ Ff $50$ Gf $50$ 7UXH FRYDULDQFH VHTXHQFHV Df $50$O Ef $50$ Ff $50$ Gf $50$ &RYDULDQFH VHTXHQFH HVWLPDWLRQ ZLWK 1 fÂ§ DQG 0 7KH PHDQVTXDUHG HUURUV 06(Vf RI WKH FRYDULDQFH HVWLPDWHV QRUPDOL]HG ZLWK UHVSHFW WR Uf DUH EDVHG RQ LQGHSHQGHQW UHDOL]DWLRQV Df $50$O Ef $50$ Ff $50$ Gf $50$ 7KH $5 FRHIILFLHQW HVWLPDWLRQ RI WKH $50$ VLJQDOV YLD WKH RYHUGHn WHUPLQHG PRGLILHG PAGE 11 $EVWUDFW RI 'LVVHUWDWLRQ 3UHVHQWHG WR WKH *UDGXDWH 6FKRRO RI WKH 8QLYHUVLW\ RI )ORULGD LQ 3DUWLDO )XOILOOPHQW RI WKH 5HTXLUHPHQWV IRU WKH 'HJUHH RI 'RFWRU RI 3KLORVRSK\ 0$7&+('),/7(5%$1. $3352$&+(6 72 63(&75$/ $1$/<6,6 $1' 3$5$0(7(5 (67,0$7,21 %\ +RQJELQ /L 0D\ &KDLUPDQ -LDQ /L 0DMRU 'HSDUWPHQW (OHFWULFDO DQG &RPSXWHU (QJLQHHULQJ 0$WFKHG),OWHUEDQN 0$),f HVWLPDWRUV UHSUHVHQW D JHQHUDO FODVV RI PHWKRGV WKDW PDNH XVH RI D VHW RI PDWFKHG ILOWHUV IRU YDULRXV HVWLPDWLRQ SXUSRVHV 7KLV GLVVHUn WDWLRQ LQYHVWLJDWHV XVLQJ 0$), DSSURDFKHV IRU FRPSOH[ VSHFWUDO DQDO\VLV DPSOLWXGH HVWLPDWLRQ IRU VLQXVRLGDO VLJQDOV DQG FRYDULDQFH VHTXHQFH HVWLPDWLRQ )RU FRPSOH[ VSHFWUDO DQDO\VLV ZH VKRZ WKDW WKH ZLGHO\ XVHG &DSRQ DQG WKH UHn FHQWO\ LQWURGXFHG $3(6 HVWLPDWRUV DUH ERWK PHPEHUV RI WKH 0$), DSSURDFK WKRXJK QHLWKHU ZDV RULJLQDOO\ GHULYHG LQ WKH 0$), IUDPHZRUN :H SURYH WKDW WR ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQ &DSRQ LV ELDVHG GRZQZDUG ZKHUHDV $3(6 LV XQELDVHG DQG WKDW WKH ELDV RI WKH IRUZDUGEDFNZDUG &DSRQ LV RQH KDOI WKDW RI WKH IRUZDUGRQO\ &DSRQ :H DOVR VKRZ WKDW &DSRQ DQG $3(6 DUH RI VLPLODU FRPSXWDWLRQDO FRPn SOH[LWLHV DQG ERWK DUH PRUH LQYROYHG WKDQ PRVW )RXULHUEDVHG PHWKRGV HVSHFLDOO\ IRU 'LPHQVLRQDO 'f GDWD (IILFLHQW LPSOHPHQWDWLRQ VFKHPHV ZKLFK VXEVWDQWLDOO\ [L PAGE 12 UHGXFH WKH FRPSXWDWLRQDO UHTXLUHPHQW DUH SUHVHQWHG IRU WKH &DSRQ DQG $3(6 HVWLn PDWRUV )RU DPSOLWXGH HVWLPDWLRQ ZH GHVFULEH D ODUJH QXPEHU RI HVWLPDWRUV ZKLFK FDQ EH FDWHJRUL]HG DV WKH /HDVW 6TXDUHV /6f :HLJKWHG /HDVW 6TXDUHV :/6f DQG 0$), PHWKRGV :KLOH DOO WKHVH PHWKRGV DUH DV\PSWRWLFDOO\ VWDWLVWLFDOO\ HIILFLHQW WKHLU SHUIRUPDQFHV LQ ILQLWH OHQJWK RI GDWD VDPSOHV DUH TXLWH GLIIHUHQW 6SHFLILFDOO\ ZH VKRZ WKDW WKH :/6 DQG 0$), PHWKRGV RXWSHUIRUP WKH /6 PHWKRGV ZKHQHYHU WKH REVHUYDWLRQ QRLVH LV FRORUHG ZH DOVR VKRZ WKDW WKH 0$), DSSURDFK LV YHU\ JHQHUDO DQG LQFOXGHV WKH :/6 DSSURDFK DV D VSHFLDO FDVH )RU FRYDULDQFH VHTXHQFH HVWLPDWLRQ ZH SUHVHQW D &DSRQ FRYDULDQFH HVWLPDWRU E\ )RXULHU LQYHUWLQJ WKH &DSRQ 3RZHU 6SHFWUDO 'HQVLW\ 36'f HVWLPDWHV :H GHVFULEH WKH )RXULHU LQYHUVLRQ LQ ERWK DQ H[DFW DQG DSSUR[LPDWH ZD\V RI ZKLFK WKH ODWWHU LV FRPSXWDWLRQDOO\ PRUH DWWUDFWLYH EXW ZLWK VRPH PLQRU DFFXUDF\ ORVV :H VKRZ WKDW WKH &DSRQ FRYDULDQFH VHTXHQFH HVWLPDWHV DUH LQ JHQHUDO EHWWHU WKDQ WKH ZLGHO\ XVHG VDPSOH FRYDULDQFH VHTXHQFHV [LL PAGE 13 &+$37(5 ,1752'8&7,21 7KLV GLVVHUWDWLRQ LV FRQFHUQHG ZLWK VSHFWUDO DQDO\VLV DQG SDUDPHWHU HVWLPDWLRQ XVLQJ 0$WFKHG),OWHUEDQN 0$),f DSSURDFKHV 7KLV FKDSWHU VHUYHV DV D JHQHUDO LQWURGXFWLRQ WR WKH EDFNJURXQG DQG VFRSH RI WKH ZRUN 6LJQLILFDQFH DQG FRQWULEXWLRQV DUH DOVR KLJKOLJKWHG LQ WKLV FKDSWHU %DFNJURXQG DQG 6FRSH RI WKH :RUN &DSRQ $3(6 DQG 0$), 6SHFWUDO (VWLPDWRUV 6SHFWUDO HVWLPDWLRQ LV DQ LPSRUWDQW GDWD DQDO\VLV WRRO ZKLFK KDV IRXQG DSSOLFDn WLRQV LQ PDQ\ GLYHUVH ILHOGV LQFOXGLQJ VSHHFK DQDO\VLV WHOHFRPPXQLFDWLRQV UDGDU DQG VRQDU V\VWHPV ELRPHGLFDO DQG VHLVPLF VLJQDO SURFHVVLQJ DQG HFRQRPLFV >@ 7KHUH DUH WZR EURDG FODVVHV RI DSSURDFKHV WR VSHFWUDO DQDO\VLV QRQSDUDPHWULF DSSURDFKHV DQG SDUDPHWULF DSSURDFKHV :KLOH QRQSDUDPHWULF PHWKRGV W\SLFDOO\ SRVWXODWH QR PRGHO IRU WKH REVHUYHG GDWD SDUDPHWULF DSSURDFKHV GR DVVXPH VRPH PRGHO VR WKDW WKH VSHFWUXP LV UHSUHVHQWHG E\ D VHW RI SDUDPHWHUV WKHUHE\ UHGXFLQJ WKH VSHFWUDO HVWLPDWLRQ SUREOHP WR WKDW RI HVWLPDWLQJ WKH SDUDPHWHUV RI WKH PRGHO 3DUDPHWULF PHWKRGV PD\ RIIHU PRUH DFFXUDWH VSHFWUDO HVWLPDWHV WKDQ QRQSDUDPHWULF PHWKRGV SURYLGHG WKDW WKH GDWD LQGHHG REVHUYH WKH PRGHO DVVXPHG E\ WKH IRUPHU ,Q WKH PRUH OLNHO\ FDVHV ZKHUH DQ DFFXUDWH GDWD PRGHO LV QRW DYDLODEOH SDUDPHWULF PHWKRGV ZLOO EH VHQVLWLYH WR PRGHO PLVPDWFK KHQFH XVLQJ WKH UREXVW QRQSDUDPHWULF DSSURDFKHV LQ VXFK FDVHV PD\ EH D EHWWHU FKRLFH 5HFHQW VWXGLHV LQ WKH OLWHUDWXUH VKRZ WKDW WKHUH KDV EHHQ UHQHZHG LQWHUHVW LQ QRQSDUDPHWULF DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ PAGE 14 )LOWHUEDQN DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ EHORQJ WR WKH FODVV RI QRQSDUD PHWULF VSHFWUDO HVWLPDWRUV $Q LPSRUWDQW PHPEHU RI ILOWHUEDQN DSSURDFKHV LV WKH &DSRQ VSHFWUDO HVWLPDWRU >@ >@ 8QOLNH WKH FODVVLFDO )RXULHUEDVHG PHWKRGV ZKLFK DUH GDWDLQGHSHQGHQW WKH &DSRQ VSHFWUDO HVWLPDWRU DGDSWV WR WKH SURFHVVHG GDWD LQ D PDQQHU VR WKDW WKH QRLVH FRPSRQHQWV RI WKH GDWD DUH UHMHFWHG VXEVWDQWLDOO\ 'XULQJ WKH SDVW IHZ GHFDGHV WKH &DSRQ VSHFWUDO HVWLPDWRU KDV EHHQ ZLGHO\ XVHG EHFDXVH LWV KLJKHU UHVROXWLRQ DQG ORZHU VLGHOREHV JLYH LW DQ DGYDQWDJH RYHU WKH )RXULHUEDVHG PHWKRGV $GGLWLRQDOO\ LWV UREXVWQHVV DQG OHVV YDULDELOLW\ PDNH LW SUHIHUDEOH WR WKH SDUDPHWULF PHWKRGV >@ >@ ,Q D UHFHQW VWXG\ LW ZDV HPSLULFDOO\ REVHUYHG WKDW XVLQJ WKH &DSRQ HVWLPDWRU IRU FRPSOH[ VSHFWUDO HVWLPDWLRQ JLYHV ELDVHG VSHFWUDO HVWLPDWHV ZKHUHDV WKH QHZO\ LQWURGXFHG $3(6 $PSOLWXGH DQG 3KDVH (VWLPDWLRQf PHWKRG DSSHDUV WR EH XQELDVHG >@ VHH )LJXUHV Df WR Gf IRU SHUIRUPDQFH GLIIHUHQFHV RI XVLQJ WKH )RXULHUEDVHG PHWKRGV &DSRQ DQG $3(6 IRU VSHFWUDO HVWLPDWLRQf 7KH IDFW WKDW ERWK &DSRQ DQG $3(6 PDNH XVH RI D VHW RI ILQLWH LPSXOVH UHVSRQVH ),5f PDWFKHG ILOWHUV ZDV REVHUYHG LQ >@ $ QXPEHU RI UHVXOWV RQ WKH SHUIRUPDQFH GLIIHUHQFHV RI WKH &DSRQ DQG $3(6 HVWLPDWRUV ZHUH DOVR UHSRUWHG WKHUHLQ +RZHYHU WKH VWXG\ LQ >@ ZDV VRPHZKDW OLPLWHG VLQFH LW RQO\ FRQVLGHUHG WKH IRUZDUGRQO\ )2f &DSRQ DQG $3(6 HVWLPDWRUV 2ZLQJ WR WKH JHQHUDO EHOLHI WKDW IRUZDUGEDFNZDUG )%f DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ XVXDOO\ SURYLGH PRUH DFFXUDWH UHVXOWV DQG DUH XVHG PRUH RIWHQ WKDQ WKHLU )2 FRXQWHUSDUWV LW LV RI LQWHUHVW WR FRQGXFW D VWXG\ WKDW LQYHVWLJDWHV KRZ WKH )% &DSRQ DQG $3(6 VSHFWUDO HVWLPDWRUV SHUIRUP ZKHQ FRPSDUHG ZLWK RQH DQRWKHU DV ZHOO DV ZLWK WKHLU )2 FRXQWHUSDUWV ,Q VRPH VHQVH ILOWHUEDQN DSSURDFKHV UHGXFH WKH SUREOHP RI VSHFWUDO HVWLPDWLRQ WR D ILOWHU GHVLJQ SUREOHP VXEMHFW WR VRPH FRQVWUDLQWV >@ ,Q 6HFWLRQ ZH GLVFXVV D JHQHUDO UXOH IRU FKRRVLQJ WKH LPSXOVH UHVSRQVHV RI WKH ILOWHUEDQN VR WKDW WKH VLJQDOWR QRLVH UDWLRV 615Vf DW WKH RXWSXWV RI WKH ILOWHUEDQN DUH PD[LPL]HG ,W IROORZV WKDW WKH PAGE 15 Ff Gf )LJXUH 6\QWKHWLF $SHUWXUH 5DGDU 6$5f LPDJHV RI D VLPXODWHG 0,* DLUSODQH REWDLQHG E\ XVLQJ Df ))7 Ef ZLQGRZHG ))7 Ff &DSRQ DQG Gf $3(6 PAGE 16 UHVXOWLQJ ILOWHUV DUH PDWFKHG ILOWHUV DQG ZH UHIHU WR WKLV FODVV RI DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ DV 0$WFKHG )OOWHUEDQN 0$),f DSSURDFKHV :H VKRZ WKDW ERWK &DSRQ DQG $3(6 DUH PHPEHUV RI 0$), DSSURDFKHV DOWKRXJK QRQH RI WKHP ZDV RULJLQDOO\ GHULYHG LQ WKH 0$), IUDPHZRUN VHH >@ >@ IRU WKHLU RULJLQDO GHULYDWLRQVf 0$), DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ PD\ DOVR EH XVHG WR GHYLVH QHZ VSHFWUDO HVWLPDWRUV (YHQ WKRXJK ZH VKRZ LQ 6HFWLRQ WKDW D UHDVRQDEOH LPSOHPHQWDWLRQ RI D VHHPLQJO\ QRYHO 0$), VSHFWUDO HVWLPDWRU UHGXFHV EDFN WR $3(6 DV ZHOO LW UHPDLQV DQ RSHQ LVVXH ZKHWKHU RWKHU LQWHUHVWLQJ 0$), VSHFWUDO HVWLPDWRUV H[LVW 7KH 0$), LQWHUSUHWDWLRQ DOVR SURYLGHV LQVLJKWV LQWR WKH &DSRQ DQG $3(6 HVWLPDWRUV DQG WKH UHODWLRQVKLS EHWZHHQ WKHP 6SHFLILFDOO\ LQ WKH IUDPHZRUN RI 0$), DSSURDFKHV ZH VKRZ E\ PHDQV RI D KLJKHURUGHU H[SDQVLRQ WHFKQLTXH LQ 6HFn WLRQ WKDW WKH RQHGLPHQVLRQDOf &DSRQ HVWLPDWRU LQGHHG XQGHUHVWLPDWHV WKH WUXH FRPSOH[ VSHFWUXP ZKLOH WKH $3(6 LV XQELDVHG WR ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQf ZH DOVR VKRZ WKDW WKH ELDV RI WKH )% &DSRQ LV RQH KDOI WKDW RI WKH )2 &DSRQ )XUWKHUPRUH ZH VKRZ WKDW WKHVH UHVXOWV FDQ EH H[WHQGHG WR WZRGLPHQVLRQDOf &DSRQ DQG $3(6 HVWLPDWRUV (IILFLHQW ,PSOHPHQWDWLRQ )RU DSSOLFDWLRQV D PDMRU FRQFHUQ RI XVLQJ &DSRQ RU $3(6 LV WKHLU FRPn SXWDWLRQDO ORDGV ZKLFK LQ JHQHUDO DUH PXFK KHDYLHU WKDQ WKRVH RI WKH WUDGLWLRQDO )RXULHUEDVHG DSSURDFKHV )RU D 6$5 LPDJH RI VL]H [ IRUPHG IURP D [ GDWD PDWUL[ DV WKRVH LQ )LJXUH WKH QXPEHU RI IORSV UHTXLUHG E\ &DSRQ DQG $3(6 LPSOHPHQWHG LQ WKH FRQYHQWLRQDO ZD\ DUH [ DQG UHVSHFWLYHO\ [ WLPHV WKDW RI WKRVH E\ WKH )RXULHUEDVHG PHWKRGV ,W VKRXOG EH PHQWLRQHG WKDW D 6$5 LPDJH RI [ LV UHODWLYHO\ VPDOO $V WKH VL]H RI WKH LPDJH LQFUHDVHV WKH DPRXQW RI FRPSXWDWLRQ E\ &DSRQ RU $3(6 LQFUHDVHV GUDVWLFDOO\ ,Q &KDSWHU ZH DGGUHVV WKH LVVXH RI KRZ WR HIILFLHQWO\ LPSOHPHQW &DSRQ DQG $3(6 IRU VSHFWUDO HVWLPDWLRQ PAGE 17 $PSOLWXGH (VWLPDWLRQ $QRWKHU SUREOHP UHODWHG WR FRPSOH[ VSHFWUDO HVWLPDWLRQ LV DPSOLWXGH HVWLPDn WLRQ IRU VLQXVRLGDO VLJQDOV ZKHUH LW LV DVVXPHG WKDW WKH QXPEHU DQG IUHTXHQFLHV RI WKH VLQXVRLGV DUH NQRZQ D SULRUL 7KH REVHUYDWLRQ QRLVH LV VWDWLRQDU\ DQG PD\EH FRORUHG ,Q WKH FDVH WKDW WKH QRLVH FDQ EH PRGHOHG H[DFWO\ WKH 0D[LPXP /LNHOLKRRG 0/f PHWKRGRORJ\ FDQ EH XVHG DQG WKH 0/ DPSOLWXGH HVWLPDWHV DUH VWDWLVWLFDOO\ HIILFLHQW +RZHYHU DQ H[DFW PRGHO RI WKH REVHUYDWLRQ QRLVH LV XVXDOO\ QRW DYDLODEOH DQG PRUHn RYHU 0/ PHWKRGV DUH LQ JHQHUDO YHU\ VHQVLWLYH WR LQDFFXUDWH PRGHO LQIRUPDWLRQ $V VXFK LW PD\ EH D EHWWHU FKRLFH WR XVH PHWKRGV WKDW GR QRW PRGHO WKH QRLVH H[DFWO\ ,Q &KDSWHU ZH GHVFULEH D UHODWLYHO\ ODUJH QXPEHU RI DPSOLWXGH HVWLPDWRUV ZKLFK DVVXPH QR PRGHO H[FHSW IRU VWDWLRQDULW\ IRU WKH REVHUYDWLRQ QRLVH 7KH DPn SOLWXGH HVWLPDWRUV FDQ EH FDWHJRUL]HG DV WKUHH JHQHUDO FODVVHV QDPHO\ /HDVW 6TXDUHV /6f :HLJKWHG /HDVW 6TXDUHV :/6f DQG 0$), DSSURDFKHV WR DPSOLWXGH HVWLPDn WLRQ ZKLFK DUH DOO DV\PSWRWLFDOO\ VWDWLVWLFDOO\ HIILFLHQW )RU ILQLWH OHQJWK RI GDWD VDPSOHV KRZHYHU WKHLU EHKDYLRUV DUH TXLWH GLIIHUHQW :H VKRZ WKDW XQGHU FHUWDLQ FLUFXPVWDQFHV WKH 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ LV HTXLYDOHQW WR WKH :/6 DSSURDFK DQG \HW WKH IRUPHU LV PRUH JHQHUDO DQG LQFOXGHV WKH ODWWHU DV D VSHFLDO FDVH :H DOVR VKRZ WKDW WKH 0$), DQG :/6 PHWKRGV LQ JHQHUDO JLYH PRUH DFFXUDWH DPn SOLWXGH HVWLPDWHV WKDQ WKH /6 PHWKRGV &RYDULDQFH 6HTXHQFH (VWLPDWLRQ 7KH IDFW WKDW FRYDULDQFH WKH WHUPV FRYDULDQFH DQG DXWRFRUUHODWLRQ DUH XVHG LQWHUFKDQJHDEO\ ZLWK RQH DQRWKHU LQ WKLV VWXG\f IXQFWLRQ DQG 3RZHU 6SHFWUDO 'HQn VLW\ 36'f DUH D )RXULHU WUDQVIRUP SDLU PDNHV WKH SUREOHP RI FRYDULDQFH VHTXHQFH HVWLPDWLRQ D UHVHDUFK WRSLF WKDW LV FORVHO\ UHODWHG WR VSHFWUDO HVWLPDWLRQ 6DPSOH FRYDULDQFH VHTXHQFHV KDYH EHHQ ZLGHO\ XVHG LQ VLJQDO SURFHVVLQJ EHFDXVH RI LWV FRPn SXWDWLRQDO VLPSOLFLW\ DQG LWV FRQVLVWHQF\ VXSSRVLQJ WKDW WKH JLYHQ VLJQDOV DUH HUJRGLF PAGE 18 WR WKH VHFRQG RUGHU :KHQ RQO\ D ILQLWH QXPEHU RI VDPSOHV DUH DYDLODEOH XVLQJ VDPn SOH FRYDULDQFH VHTXHQFHV LPSOLHV WKDW WKH GDWD EH\RQG WKH REVHUYHG GXUDWLRQ HLWKHU LV ]HUR RU UHSHDWV LWVHOI SHULRGLFDOO\ ZKLFK LV FHUWDLQO\ QRW D UHDOLVWLF DVVXPSWLRQ 7KHUH KDYH EHHQ VHYHUDO DWWHPSWV LQ WKH OLWHUDWXUH WR GHULYH RWKHU FRYDULDQFH HVWLPDWRUV VHH &KDSWHU IRU VRPH GHWDLOVf ,W LV NQRZQ WKDW WKH VDPSOH FRYDULDQFH VHTXHQFH DQG WKH GDWD SHULRGRJUDP FRQVWLWXWH D )RXULHU WUDQVIRUP SDLU ,W LV DOVR NQRZQ WKDW WKH SH ULRGRJUDP LV D VWDWLVWLFDOO\ LQHIILFLHQW LQ IDFW LQFRQVLVWHQWf 36' HVWLPDWRU >@ 7KLV REVHUYDWLRQ VXJJHVWV WKDW EHWWHU FRYDULDQFH HVWLPDWRUV PD\ EH REWDLQHG E\ )RXULHU LQYHUWLQJ EHWWHU 36' HVWLPDWRUV 6LQFH WKH 0$), 36' HVWLPDWHV DUH LQ JHQHUDO PRUH DFFXUDWH WKDQ SHULRGRJUDPV ZH SURSRVH D QHZ FRYDULDQFH VHTXHQFH HVWLPDWRU E\ )RXULHU LQYHUWLQJ WKH 0$), 36' HVWLPDWHV 6SHFLILFDOO\ ZH PDNH XVH RI WKH &DSRQ 36' DQG VXFK DQ DSSURDFK LV UHIHUUHG WR DV WKH &DSRQ FRYDULDQFH VHTXHQFH HVWLPDWRU 7KH VDPH PHWKRGRORJ\ FDQ EH VLPLODUO\ DSSOLHG WR WKH $3(6 36' HVWLPDWHV WKRXJK WKH $3(6 FRYDULDQFH HVWLPDWHV DUH XVXDOO\ VLPLODU WR WKH &DSRQ FRYDULDQFH HVWLPDWHV 7KH UHDVRQ LV WKDW GHVSLWH WKHLU GLIIHUHQW SHUIRUPDQFH IRU FRPSOH[ VSHFWUDO HVWLPDn WLRQ WKH &DSRQ DQG $3(6 36' HVWLPDWRUV XVXDOO\ EHKDYH TXLWH VLPLODUO\ HVSHFLDOO\ IRU FRQWLQXRXV VSHFWUD DOVR VHH WKH GLVFXVVLRQV LQ 6HFWLRQ f ,W LV REVHUYHG WKDW &DSRQ FRYDULDQFH HVWLPDWHV DUH LQ JHQHUDO EHWWHU WKDQ VDPSOH FRYDULDQFH HVWLPDWHV LQ WHUPV RI PHDQVTXDUHG HUURUV 06(Vf 6LJQLILFDQFH DQG &RQWULEXWLRQV 7KH PDLQ UHVXOWV RI WKLV GLVVHUWDWLRQ DUH DV IROORZV :H LQWURGXFH D QHZ JHQHUDO FODVV RI 0$), DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ :H VKRZ WKDW WKH &DSRQ DQG $3(6 HVWLPDWRUV WKRXJK RULJLQDOO\ QRW GHULYHG LQ WKH 0$), IUDPHZRUN DUH ERWK PHPEHUV RI WKH 0$), FODVV 7R ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQ ZH SURYH WKDW WKH &DSRQ HVWLPDWRU JLYHV ELDVHG GRZQZDUGf FRPSOH[ VSHFWUDO HVWLPDWHV ZKHUHDV WKH $3(6 PHWKRG PAGE 19 LV XQELDVHG ZH DOVR SURYH WKDW WKH ELDV IRU WKH )% &DSRQ LV RQH KDOI WKDW RI WKH )2 &DSRQ 7KHVH WKHRUHWLFDO UHVXOWV VXSSOHPHQWHG ZLWK WKH HPSLULFDO REVHUn YDWLRQ WKDW &DSRQ XVXDOO\ XQGHUHVWLPDWHV WKH VSHFWUXP LQ VDPSOHV RI SUDFWLFDO OHQJWK ZKLOH $3(6 LV QHDUO\ XQELDVHG DUH EHOLHYHG WR SURYLGH FRPSHOOLQJ UHDn VRQV IRU SUHIHUULQJ $3(6 WR &DSRQ LQ PRVW SUDFWLFDO DSSOLFDWLRQV :H SUHVHQW HIILFLHQW LPSOHPHQWDWLRQ WHFKQLTXHV IRU WKH 0$), VSHFWUDO HVWLn PDWRUV :H VKRZ WKDW E\ XVLQJ VXFK WHFKQLTXHV WKH DPRXQW RI FRPSXWDWLRQ UHTXLUHG E\ WKH &DSRQ RU $3(6 HVWLPDWRU LV VLJQLILFDQWO\ UHGXFHG 7KH 0$), LGHD LV DOVR H[WHQGHG IRU DPSOLWXGH HVWLPDWLRQ IRU VLQXVRLGDO VLJQDOV LQ FRORUHG QRLVH 6SHFLILFDOO\ ZH PDNH H[WHQVLRQV WR WKH &DSRQ DQG $3(6 DOJRn ULWKPV WR PXOWLSOH VLQXVRLGV ZLWK NQRZQ IUHTXHQFLHV )XUWKHUPRUH ZH GHVFULEH D JHQHUDOL]HG 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ IRU PXOWLSOH VLQXVRLGV $ QHZ FRYDULDQFH VHTXHQFH HVWLPDWRU LV SUHVHQWHG E\ )RXULHU LQYHUWLQJ WKH &DSRQ VSHFWUDO HVWLPDWHV 7KH &DSRQ FRYDULDQFH VHTXHQFH HVWLPDWHV DUH VKRZQ WR EH PRUH DFFXUDWH WKDQ WKH FRQYHQWLRQDO VDPSOH FRYDULDQFH VHTXHQFH HVWLn PDWHV 2UJDQL]DWLRQ RI WKH 'LVVHUWDWLRQ 7KH GLVVHUWDWLRQ LV RUJDQL]HG DV IROORZV &KDSWHU JLYHV D OLWHUDWXUH VXUYH\ RI VXFK WRSLFV DV ILOWHUEDQN DSSURDFKHV WR VSHFWUDO DQDO\VLV WKH &DSRQ VSHFWUDO HVWLPDn WRU DQG LWV DSSOLFDWLRQV DQG FRYDULDQFH HVWLPDWLRQ ,Q &KDSWHU ZH ILUVW LQWURGXFH WKH 0$), DSSURDFK WR FRPSOH[ VSHFWUDO HVWLPDWLRQ :H QH[W VKRZ WKDW WKH &DSRQ DQG $3(6 HVWLPDWRUV DUH ERWK PHPEHUV RI WKH 0$), DSSURDFK IROORZHG E\ FRPn SXWDWLRQDO DQG VWDWLVWLFDO DQDO\VHV RI WKH 0$), VSHFWUDO HVWLPDWRUV ([WHQVLRQV WR WKH FDVH DUH DOVR LQFOXGHG &KDSWHU DGGUHVVHV WKH LPSOHPHQWDWLRQ LVVXH RI WKH 0$), VSHFWUDO HVWLPDWRUV &KDSWHU GLVFXVVHV DPSOLWXGH HVWLPDWLRQ IRU VLQXVRLGDO PAGE 20 VLJQDOV IURP REVHUYDWLRQ FRUUXSWHG E\ FRORUHG QRLVH 7KH 0$), FRQFHSW LQWURGXFHG LQ &KDSWHU LV H[WHQGHG IRU DPSOLWXGH HVWLPDWLRQ ,Q &KDSWHU ZH LQYHVWLJDWH FRYDULn DQFH VHTXHQFH HVWLPDWLRQ XVLQJ WKH &DSRQ VSHFWUDO HVWLPDWHV )LQDOO\ ZH VXPPDUL]H WKLV ZRUN DQG RXWOLQH IXWXUH ZRUN LQ &KDSWHU PAGE 21 &+$37(5 /,7(5$785( 6859(< +LVWRULFDO DQG PRGHUQ SHUVSHFWLYHV RQ WKH JHQHUDO WRSLFV RI VSHFWUDO DQDO\VLV DQG SDUDPHWHU HVWLPDWLRQ KDYH EHHQ ZHOO GRFXPHQWHG LQ WKH OLWHUDWXUH >@ >@ 0DQ\ FODVVLFDO DUWLFOHV ERWK WKHRUHWLFDO DQG DSSOLFDWLRQRULHQWHG KDYH EHHQ UHSULQWHG >@ >@ ([FHOOHQW WH[WV DUH DOVR DYDLODEOH >@ >@ >@ ,Q WKLV FKDSWHU ZH JLYH D EULHI UHYLHZ RI D QXPEHU RI VXEMHFWV WKDW DUH UHODWHG WR RXU ZRUN QDPHO\ ILOWHUEDQN DSn SURDFKHV WR VSHFWUDO HVWLPDWLRQ WKH &DSRQ PHWKRG DQG DSSOLFDWLRQV DQG FRYDULDQFH HVWLPDWLRQ )LOWHUEDQN $SSURDFKHV DQG &DSRQ (VWLPDWRU 8QOHVV WKH REVHUYHG VLJQDO FDQ EH PRGHOHG ZLWK D ILQLWH QXPEHU RI SDUDPHn WHUV HVWLPDWLQJ WKH VSHFWUXP RI D VLJQDO EDVHG RQ D ILQLWH OHQJWK RI REVHUYDWLRQV LV DQ LOOSRVHG SUREOHP IURP D VWDWLVWLFDO VWDQGSRLQW VLQFH ZH DUH UHTXLUHG WR HVWLPDWH DQ LQILQLWH QXPEHU RI LQGHSHQGHQW VSHFWUDO YDOXHV EDVHG RQ D ILQLWH QXPEHU RI VDPn SOHV $Q DVVXPSWLRQ PDGH E\ ILOWHUEDQN DSSURDFKHV DQG PRVW RWKHU QRQSDUDPHWULF PHWKRGV LV WR DVVXPH WKDW WKH 36' RI WKH REVHUYHG VLJQDO LV QHDUO\f FRQVWDQW RYHU D QDUURZEDQG DURXQG DQ\ JLYHQ IUHTXHQF\ 1DWXUDOO\ RQH FDQ SURFHHG E\ SDVVLQJ WKH REVHUYHG VLJQDO WKURXJK D EDQGSDVV ILOWHU ZKLFK LV VZHSW WKURXJK WKH IUHTXHQF\ EDQG RI LQWHUHVW DQG HVWLPDWLQJ WKH FRPSOH[ DPSOLWXGH LI FRPSOH[ VSHFWUDO HVWLPDn WLRQ LV RI LQWHUHVW RU PHDVXULQJ WKH ILOWHU RXWSXW SRZHU DQG GLYLGLQJ LW E\ WKH ILOWHU EDQGZLGWK LI 36' HVWLPDWLRQ LV WKH GHVLUHG JRDOf D SURFHGXUH DGRSWHG E\ DOO ILOWHU EDQN DSSURDFKHV 2EYLRXVO\ KRZ WR FKRRVH WKH QDUURZEDQG ILOWHUV LV D FULWLFDO LVVXH RI ILOWHUEDQN DSSURDFKHV (YHQ WKRXJK VRPH RI WKH FODVVLFDO )RXULHUEDVHG PHWKRGV PAGE 22 VXFK DV SHULRGRJUDP E\ 6FKXVWHU >@ DQG LWV YDULRXV YDULDWLRQV LQFOXGLQJ WKH IDPRXV %ODFNPDQ7XFNH\fV PHWKRG >@ FDQ DOVR EH FDVW LQ WKH IUDPHZRUN RI ILOWHUEDQN DSn SURDFKHV RULJLQDOO\ WKH\ ZHUH QRW GHVLJQHG LQ VXFK D PDQQHU ,Q RWKHU ZRUGV WKRVH PHWKRGV PDGH QR DWWHPSW WR SXUSRVHO\ GHVLJQ D JRRG EDQGSDVV ILOWHU WR DFKLHYH VRPH GHVLUHG FKDUDFWHULVWLFV $ QRWDEOH H[DPSOH RI ILOWHUEDQN DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ LV WKH 5( ),/ 5(ILQHG ),/WHUf PHWKRG ZKLFK ZDV ILUVW LQWURGXFHG LQ >@ DQG ZDV IXUWKHU GHYHORSHG LQ >@ DOVR GLVFXVVHG LQ >@ >@ >@f 7KH 5(),/ PHWKRG LV FORVH LQ VSLULW WR WKH 'DQLHOO DSSURDFK >@ WR UHGXFLQJ WKH YDULDQFH RI WKH SHULRGRJUDP 7KDW LV 5(),/ GRHV QRW VSOLW WKH DYDLODEOH VDPSOHV LQ VKRUWHU VWUHWFKHV 7KH 5(),/ LGHD LV WR GHVLJQ D EDQN RI ILOWHUV ZKLFK SDVV WKH VLJQDO FRPSRQHQWV ZLWKLQ WKH SDVVEDQGV DV PXFK DV SRVVLEOH UHODWLYH WKH WRWDO SRZHU DQG LQ WKH PHDQ WLPH DWWHQXDWH WKH IUHTXHQFLHV RXWVLGH WKH FRUUHVSRQGLQJ SDVVEDQGV ,W WXUQV RXW WKDW 5(),/ LQ JHQHUDO JLYHV EHWWHU VSHFWUDO HVWLPDWHV WKDQ WKH WUDGLWLRQDO )RXULHUEDVHG PHWKRGV 1HYHUWKHn OHVV WKH 5(),/ ILOWHUV VKDUH D FRPPRQ FKDUDFWHULVWLF ZLWK WKH )RXULHUEDVHG RQHV WKH\ DUH DOO GDWDLQGHSHQGHQW LQ WKH VHQVH WKDW WKH\ GR QRW DGDSW WR WKH UHFHLYHG GDWD LQ DQ\ ZD\ 3UHVXPDEO\ LW ZRXOG EH EHQHILFLDO WR WDNH WKH GDWD SURSHUWLHV LQWR DFFRXQW ZKHQ GHVLJQLQJ ILOWHUEDQNV 7KH &DSRQ PHWKRG LV RQH DQG SHUKDSV WKH PRVW IDPRXV RQHf RI VXFK GDWD GHSHQGHQW ILOWHUEDQN VSHFWUDO HVWLPDWRUV 7KH &DSRQ HVWLPDWRU ZDV ILUVW QDPHG DV WKH 0D[LPXP /LNHOLKRRG 0HWKRG 0/0f GXH WR WKH WKH 0/ DQG *DXVVLDQ SURFHVV FRQWH[W XVHG LQ &DSRQfV RULJLQDO ZRUN >@ ,W WXUQV RXW WKDW 0/0 LV D PLVQRPHU VLQFH LW LV QRW DQ 0/ VSHFWUDO HVWLPDWRU DQG LW GRHV QRW SRVVHVV DQ\ RI WKH SURSHUWLHV RI DQ 0/ HVWLPDWRU ,Q >@ DQG >@ &DSRQfV PHWKRG ZDV UHIHUUHG WR DV WKH 0LQLPXP 9DULDQFH 6SHFWUDO (VWLPDWRU 096(f EHFDXVH LW LV GHULYHG E\ PLQLPL]LQJ WKH YDULDQFH RI WKH RXWSXW RI D QDUURZEDQG ILOWHU (YHQ WKH QDPH 096( LV LQDFFXUDWH LQ WKH VHQVH PAGE 23 WKDW &DSRQ VSHFWUDO HVWLPDWHV GR QRW SRVVHVV WKH PLQLPXP YDULDQFH SURSHUW\ ,Q WKLV VWXG\ LW LV VLPSO\ QDPHG WKH &DSRQ VSHFWUDO HVWLPDWRU 7KH &DSRQ VSHFWUDO HVWLPDWRU ZDV RULJLQDOO\ LQWURGXFHG E\ &DSRQ >@ >@ IRU DSSOLFDWLRQV LQ PXOWLGLPHQVLRQDO VHLVPLF DUUD\ IUHTXHQF\ZDYHQXPEHU DQDO\VLV ,W LV UHIRUPXODWHG E\ /DFRVV >@ IRU DSSOLFDWLRQV WR WLPHVHULHV SUREOHPV 7KH FRQVWUDLQWV DGRSWHG E\ WKH &DSRQ PHWKRG DUH VXFK WKDW WKH VLJQDO DW WKH FXUUHQW IUHTXHQF\ LV SDVVHG XQGLVWRUWHG ZLWK XQLW JDLQf ZKLOH WKH RXWSXW SRZHU RI WKH RYHUn DOO IUHTXHQF\ GRPDLQ LV PLQLPL]HG $ QXPEHU RI UHVHDUFKHV KDYH EHHQ FRQGXFWHG WR VWXG\ WKH &DSRQ VSHFWUDO HVWLPDWRU 6SHFLILFDOO\ &DSRQ DQG *RRGPDQ GHPRQn VWUDWHG >@ >@ WKDW WKH &DSRQ VSHFWUXP KDV D PHDQ DQG D YDULDQFH WKDW EHKDYHV OLNH WKH DYHUDJHG SHULRGRJUDP WKDW LV &DSRQ VSHFWUD DUH XVXDOO\ OHVV YDULDEOH WKDQ SHULRGRJUDPV $GGLWLRQDOO\ /DFRVVfV HPSLULFDO VWXG\ >@ VXJJHVWHG WKDW WKH UHVROXn WLRQ RI WKH &DSRQ PHWKRG LV EHWZHHQ WKDW RI %XUJfV $XWR5HJUHVVLYH $5f VSHFWUDO HVWLPDWRU >@ >@ DQG WKDW RI WKH SHULRGRJUDP /DFRVVfV VWXG\ DOVR VXJJHVWHG WKDW WKH VWDWLVWLFDO YDULDELOLW\ RI &DSRQ LV OHVV WKDQ WKDW RI WKH $5 HVWLPDWRU %XUJ ODWHU SURYHG WKDW WKH UHFLSURFDO RI WKH &DSRQ VSHFWUXP RI RUGHU 0 WKH OHQJWK RI WKH &DSRQ ILOWHU LV HTXDO WR WKH DYHUDJH RI WKH UHFLSURFDOV RI WKH $5 VSHFWUD IURP RUGHU WR 0 >@ 6XFK DQ DYHUDJLQJ HIIHFW H[SODLQV WKH HPSLULFDO REVHUYDWLRQV PDGH E\ /DFRVV )XUWKHU UHVHDUFKHV LQ TXDQWL]LQJ WKH UHVROXWLRQ SURSHUWLHV RI &DSRQ VSHFWUD ZHUH UHSRUWHG LQ >@ >@ 6LQFH &DSRQfV PHWKRG LV D EDVLF VSHFWUDO HVWLPDWRU LW PD\ EH XVHG LQ DQ\ DSn SOLFDWLRQ ZKHUH WKH VSHFWUXP RI WKH VWXGLHG VLJQDO SOD\V DQ LPSRUWDQW UROH ,QGHHG HYHU VLQFH LWV ILUVW DSSHDUDQFH WKH &DSRQ VSHFWUDO HVWLPDWRU KDV EHHQ ZLGHO\ XVHG LQ PDQ\ DUHDV LQFOXGLQJ UDGDU VRQDU FRPPXQLFDWLRQV LPDJLQJ JHRSK\VLFDO H[SORn UDWLRQ DVWURSK\VLFDO H[SORUDWLRQ DQG ELRPHGLFDO HQJLQHHULQJ VHH >@ >@ DQG WKH UHIHUHQFHV WKHUHLQf 5DWKHU WKDQ PDNLQJ D IXWLOH DWWHPSW DW PDNLQJ D IXOO GRFXPHQW PAGE 24 LQ JUHDW OHQJWK YYH GHVFULEH LQ WKH IROORZLQJ D IHZ W\SLFDO DSSOLFDWLRQV RI WKH &DSRQ HVWLPDWRU $Q LQWHUHVWLQJ DSSOLFDWLRQ RI &DSRQfV PHWKRG LV EHDPIRUPLQJ %HDPIRUPLQJ LV XVHG LQ FRQMXQFWLRQ ZLWK DQ DUUD\ RI VHQVRUV WR SURYLGH D YHUVDWLOH IRUP RI VSDn WLDO ILOWHULQJ 7KH REMHFWLYH RI EHDPIRUPLQJ LV WR HVWLPDWH WKH VLJQDO DUULYLQJ IURP D GHVLUHG GLUHFWLRQ LQ WKH SUHVHQFH RI QRLVH DQG LQWHUIHULQJ VLJQDOV ,I WKH GHVLUHG DQG LQn WHUIHULQJ VLJQDOV RFFXS\ WKH VDPH WHPSRUDOf IUHTXHQF\ EDQG WKHQ WHPSRUDO ILOWHULQJ FDQQRW EH DSSOLHG WR GLVWLQJXLVK VLJQDO IURP LQWHUIHUHQFH +RZHYHU VLQFH WKH GHVLUHG VLJQDO DQG LQWHUIHULQJ VLJQDOV W\SLFDOO\ RULJLQDWH IURP GLIIHUHQW ORFDWLRQV VXFK VSDWLDO GLYHUVLW\ FDQ EH H[SORLWHG WR VHSDUDWH VLJQDO IURP LQWHUIHUHQFH XVLQJ D VSDWLDO ILOWHU ,Q )URVW PDGH XVH RI D OLQHDU FRQVWUDLQHG RSWLPL]DWLRQ WHFKQLTXH DQG LQWURGXFHG DQ DGDSWLYH EHDPIRUPHU >@ UHIHUUHG WR DV WKH /&09 /LQHDUO\ &RQVWUDLQHG 0LQLn PXP 9DULDQFHf EHDPIRUPHU LQ WKH DUUD\ VLJQDO SURFHVVLQJ FRPPXQLW\ 7KH EDVLF LGHD RI WKH /&09 EHDPIRUPLQJ LV WR FRQVWUDLQ WKH UHVSRQVH RI WKH EHDPIRUPHU VR WKDW VLJQDOV IURP WKH GLUHFWLRQ RI LQWHUHVW DUH SDVVHG ZLWK VSHFLILHG JDLQ ZKLOH PLQLPL]LQJ WKH RXWSXW SRZHU GXH WR LQWHUIHULQJ VLJQDOV DQG QRLVH DUULYLQJ IURP RWKHU GLUHFWLRQV 2QH ZRXOG LPPHGLDWHO\ QRWLFH WKH VLPLODULW\ WR WKH FRQVWUDLQWV DGRSWHG E\ &DSRQ ,QGHHG WKH /&09 EHDPIRUPHU LV D GLUHFW H[WHQVLRQ RI WKH WHPSRUDO &DSRQ ILOWHU WR WKH VSDWLDO GRPDLQ $PRQJ WKH VRFDOOHG VWDWLVWLFDOO\ RSWLPXP EHDPIRUPHUV /&09 LV SHUKDSV WKH PRVW SRSXODU RQH VLQFH LW QHHGV QR DX[LOLDU\ FKDQQHOV DV UHTXLUHG E\ WKH 0XOWLSOH 6LGHOREH &DQFHOOHU 06&f >@ DQG XQOLNH WKH FODVV RI RSWLPXP EHDP IRUPHUV SURSRVHG E\ :LGURZ HW DO >@ ZKLFK UHTXLUH UHIHUHQFH VLJQDOV LW LV EOLQG $ XVHIXO VWUXFWXUH IRU /&09 LPSOHPHQWDWLRQ LV WKH *HQHUDOL]HG 6LGHOREH &DQFHOOHU *6&f >@ *6& UHSUHVHQWV DQ DOWHUQDWLYH IRUPXODWLRQ RI WKH /&0/ EHDPIRUPHU ZKLFK FKDQJHV WKH FRQVWUDLQHG RSWLPL]DWLRQ SUREOHP RI /&0/ WR DQ XQFRQVWUDLQHG RQH 7KH XQFRQVWUDLQHG QDWXUH OHQGV *6& WR DGDSWLYH LPSOHPHQWDWLRQ PRUH UHDGLO\ WKDQ WKH RULJLQDO /&09 EHDPIRUPHU DQG KHQFH *6& LV WKH RQH XVHG PRUH RIWHQ PAGE 25 LQ SUDFWLVH *6& DOVR IRXQG DSSOLFDWLRQV LQ GLUHFWVHTXHQFH &RGH'LYLVLRQ 0XOWLSOH $FFHVV &'0$f FRPPXQLFDWLRQ V\VWHPV IRU EOLQG PXOWLXVHU GHWHFWLRQ >@ >@ >@ $Q LPSRUWDQW H[WHQVLRQ RI WKH /&0/ EHDPIRUPHU ZDV PDGH LQ >@ >@ IRU DSSOLFDn WLRQV LQ DLUERUQH UDGDU V\VWHPV $ W\SLFDO VLJQDO HQYLURQPHQW IDFHG E\ VXFK V\VWHPV FRQVLVWV RI VWURQJ FOXWWHULQWHUIHUHQFH RI FRPSOLFDWHG DQJOH'RSSOHU VSHFWUXP ZKLFK LV XQNQRZQ DQG PD\ EH YDU\LQJ LQ ERWK WLPH DQG VSDFH 7KH H[WHQVLRQ PDGH E\ %UHQQDQ HO DO LV WR VLPXOWDQHRXVO\ FRPELQH WKH VLJQDOV UHFHLYHG RQ PXOWLSOH HOHPHQWV RI DQ DQWHQQD DUUD\ WKH VSDWLDO GRPDLQf DQG IURP PXOWLSOH SXOVH UHSHWLWLRQ SHULn RGV WKH WHPSRUDO GRPDLQf 6XFK D WHFKQLTXH LV UHIHUUHG WR DV VSDFHWLPH DGDSWLYH SURFHVVLQJ WKDW KDV UHFHLYHG PXFK UHVHDUFK LQWHUHVW UHFHQWO\ >@ >@ $QRWKHU LQWHUHVWLQJ DSSOLFDWLRQ RI WKH &DSRQ VSHFWUDO HVWLPDWRU LV 6$5 LPDJn LQJ >@ >@ &RQYHQWLRQDO 6$5 LPDJLQJ WHFKQLTXHV DUH WKH ))7 )DVW )RXULHU 7UDQVn IRUPf RU ZLQGRZHG))7 PHWKRGV $ QXPEHU RI SDUDPHWULF VSHFWUDO HVWLPDWLRQ PHWKn RGV KDYH DOVR EHHQ XVHG IRU 6$5 LPDJLQJ >@ >@ >@ WKRXJK WKHLU LQWHUHVW LQ 6$5 LPDJLQJ LV OLPLWHG EHFDXVH RI WKHLU VHQVLWLYLW\ WR PRGHO HUURUV $ FRPSDUDWLYH VWXG\ LQ >@ VKRZHG WKDW DGDSWLYH ILOWHUEDQN DSSURDFKHV VXFK DV &DSRQ RIIHUV JRRG 6$5 LPDJHV DQG HQMR\V WKH DGYDQWDJH RI UREXVWQHVV DV FRPSDUHG WR SDUDPHWULF PHWKRGV $ QXPEHU RI PRGLILHG &DSRQ PHWKRGV KDYH DOVR EHHQ VXJJHVWHG IRU 6$5 LPDJLQJ >@ >@ &RYDULDQFH (VWLPDWLRQ 2QH W\SH RI WKH FRYDULDQFH HVWLPDWLRQ SUREOHPV LV WR HVWLPDWH WKH FRYDULDQFH VHTXHQFH IURP D ILQLWH QXPEHU RI GDWD VDPSOHV $ VWDQGDUG WHFKQLTXH IRU HVWLPDWLQJ FRYDULDQFH VHTXHQFHV XVHV WKH ELDVHG RU XQELDVHG VDPSOH FRYDULDQFH HVWLPDWRU 7KH ELDVHG FRYDULDQFH HVWLPDWRU LV PRUH FRPPRQO\ XVHG EHFDXVH LW SURYLGHV VPDOOHU 06(V WKDQ WKH XQELDVHG RQH DQG JXDUDQWHHV WKH FRYDULDQFH HVWLPDWHV WR EH QRQQHJDWLYH >@ 7KH SUREOHP RI WKH VDPSOH FRYDULDQFH HVWLPDWRUV LV WKH XQUHDOLVWLF ZLQGRZLQJ PAGE 26 WKH\ DVVXPH RQ WKH REVHUYHG GDWD %\ H[SORLWLQJ WKH $5 VSHFWUDO HVWLPDWRU %XUJ SURSRVHG D WHFKQLTXH WKDW FDQ RIIHU FRYDULDQFH VHTXHQFH HVWLPDWHV IRU DQ\ GHVLUHG ODJ DQG KHQFH DYRLGV WKH ZLQGRZLQJ SUREOHP VXIIHUHG E\ WKH VDPSOH FRYDULDQFH HVWLPDWRUV >@ +RZHYHU LW ZDV VKRZQ LQ >@ WKDW WKH %XUJ FRYDULDQFH HVWLPDWHV DUH OHVV DFFXUDWH DQG PRUH YDULDEOH WKDQ WKH VDPSOH FRYDULDQFH HVWLPDWHV ,Q &KDSWHU ZH GHVFULEH D QHZ PHWKRG IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ EDVHG RQ &DSRQ VSHFWUD (VWLPDWLQJ VWUXFWXUHG FRYDULDQFH PDWUL[ RI WKH REVHUYHG GDWD YHFWRUV UHSUHn VHQWV DQRWKHU W\SH RI WKH FRYDULDQFH HVWLPDWLRQ SUREOHP )RU H[DPSOH WKH FRYDULDQFH PDWUL[ RI D VWDWLRQDU\ FRPSOH[ VLJQDO LV +HUPLWLDQ DQG 7RHSOLW] +RZHYHU WKH VDPn SOH FRYDULDQFH PDWUL[ REWDLQHG IURP D ILQLWH QXPEHU RI GDWD VDPSOHV VHOGRP KDV WKLV VWUXFWXUH 6WUXFWXUHG FRYDULDQFH PDWUL[ HVWLPDWLRQ LV RI LPSRUWDQFH LQ D YDULHW\ RI DSn SOLFDWLRQV LQFOXGLQJ DUUD\ VLJQDO SURFHVVLQJ DQG WLPH VHULHV DQDO\VLV >@ $Q LQWXLWLYH ZD\ WR REWDLQ VWUXFWXUHG FRYDULDQFH HVWLPDWHV LV WR IRUFH WKH GHVLUHG VWUXFWXUHV RQ WKH VDPSOH FRYDULDQFH PDWUL[ D PHWKRGRORJ\ DGRSWHG E\ WKH ,WHUDWHG 7RHSOLW] $SSUR[Ln PDWLRQ 0HWKRG ,7$0f >@ >@ >@ 6SHFLILFDOO\ ,7$0 DOWHUQDWLYHO\ PDNHV XVH RI UDQN DSSUR[LPDWLRQ YLD VLQJXODU YDOXH GHFRPSRVLWLRQf DQG 7RHSOLW]DWLRQ DORQJ WKH GLDJRQDOV XQWLO FRQYHUJHQFH LV UHDFKHG 2EYLRXVO\ VXFK D PHWKRG LV E\ QDWXUH KHXULVn WLF DQG QR RSWLPDOLW\ FDQ EH DVVRFLDWHG ZLWK LW WKRXJK WKH ,7$0 FRYDULDQFH PDWUL[ HVWLPDWH LV LQ JHQHUDO FORVHU WR WKH WUXH FRYDULDQFH PDWUL[ WKDQ WKH VDPSOH FRYDULDQFH PDWUL[ LQ WKH )UREHQLXV QRUP VHQVH 2SWLPXP VWUXFWXUHG FRYDULDQFH PDWUL[ HVWLPDWH PD\ EH REWDLQHG E\ PD[LPL]LQJ WKH FRUUHVSRQGLQJ OLNHOLKRRG IXQFWLRQ DV FRQVLGHUHG LQ >@ >@ >@ >@ >@ +RZHYHU VLQFH WKHUH H[LVWV QR FORVHGIRUP VROXWLRQ WR WKH FRPSOLFDWHG QRQOLQHDU 0/ HVWLPDWLRQ SUREOHP IRU +HUPLWLDQ 7RHSOLW] PDWULFHV WKH 0/ PHWKRGV SURSRVHG LQ WKHVH VWXGLHV DUH LWHUDWLYH DQG FRPSXWDWLRQDOO\ LQYROYHG DQG PRUHRYHU WKH\ DUH QRW JXDUDQWHHG WR \LHOG WKH JOREDO RSWLPDO VROXWLRQ ZKLFK WR VRPH GHJUHH OLPLWV WKH LQWHUHVW LQ XVLQJ WKH 0/ VWUXFWXUHG FRYDULDQFH PDWUL[ HVn WLPDWH LQ SUDFWLFDO DSSOLFDWLRQV $Q DSSUR[LPDWH 0/ PHWKRG WKDW PDNHV XVH RI WKH PAGE 27 ([WHQGHG ,QYDULDQFH 3ULQFLSOH (;,3f >@ ZDV UHFHQWO\ SUHVHQWHG LQ >@ >@ 7KLV PHWKRG SURYLGHV DV\PSWRWLF IRU ODUJH VDPSOHVf 0/ HVWLPDWLRQ IRU VWUXFWXUHG FRYDULn DQFH PDWULFHV $ FORVHGIRUP VROXWLRQ IRU WKH HVWLPDWLRQ +HUPLWLDQ 7RHSOLW] PDWULFHV LV REWDLQHG ZKLFK PDNHV WKH SURSRVHG PHWKRG FRPSXWDWLRQDOO\ PXFK VLPSOHU WKDQ PRVW H[LVWLQJ +HUPLWLDQ 7RHSOLW] PDWUL[ HVWLPDWLRQ DOJRULWKPV $GGLWLRQDOO\ LW ZDV DOVR VKRZQ WKDW XVLQJ WKH WHFKQLTXH LQ VXFK DUUD\ SURFHVVLQJ DOJRULWKPV DV 086,& >@ DQG (635,7 >@ PDNHV WKHP DFKLHYH WKH &UDPU5DR %RXQG &5%f IRU DQJOH HVWLPDWLRQ LH WKH EHVW SHUIRUPDQFH IRU DQ\ XQELDVHG PHWKRGV PAGE 28 &+$37(5 0$), $3352$&+ 72 63(&75$/ (67,0$7,21 )LOWHUEDQN DSSURDFKHV GHFRPSRVH WKH REVHUYDWLRQV ^\Qf`A RI D VWDWLRQDU\ VLJQDO \Wf DV >@ >@ >@ >@ \^Qf fÂ§ RÂXfHMXQ HZQf Q 1 Z*>Uf f ZKHUH DFRf GHQRWHV WKH FRPSOH[ DPSOLWXGH RI WKH VLQXVRLGDO VLJQDO ZLWK IUHTXHQF\ X! DQG HZQf GHQRWHV WKH QRLVH RU UHVLGXDOf WHUP DW IUHTXHQF\ X! DVVXPHG WR EH ]HURPHDQ 7KH SUREOHP RI LQWHUHVW LV WR HVWLPDWH DXf IRU DQ\ JLYHQ X! %ULHIO\ VWDWHG PRVW ILOWHUEDQN VSHFWUDO DSSURDFKHV DGGUHVV WKH DIRUHPHQWLRQHG SUREOHP E\ IROORZLQJ WZR PDLQ VWHSV Df SDVV WKH GDWD ^\Qf` WKURXJK D EDQGSDVV ILOWHU ZLWK YDU\LQJ FHQWHU IUHTXHQF\ X DQG Ef REWDLQ WKH HVWLPDWHV Â£Zf IRU X > Wf RI WKH FRPSOH[ DPSOLWXGH IURP WKH ILOWHUHG GDWD 7KH EDQGSDVV ILOWHU XVHG LV XVXDOO\ DQ 0WDS ),5 ILOWHU ZLWK LWV FRHIILFLHQW YHFWRU JLYHQ E\ U ?7 KZ K? K K 0 f ZKHUH f7 GHQRWHV WKH WUDQVSRVH 7KH FKRLFH RI 0 LV GLVFXVVHG LQ 6HFWLRQ f 2EVHUYH WKDW WKH QRWDWLRQ HPSKDVL]HV WKH GHSHQGHQFH RI WKH YHFWRU LQ f RQ WKH FHQWHU IUHTXHQF\ OR $OWKRXJK UXOHV IRU FKRRVLQJ KA YDU\ D UDWKHU JHQHUDO RQH IRU WKH FKRLFH RI D PDWFKHG ILOWHU LV GLVFXVVHG LQ 6HFWLRQ )RUZDUG DQG %DFNZDUG 'DWD 9HFWRUV /HW \ \ \Â f \O 0 f = / fÂ§ f PAGE 29 EH WKH RYHUODSSLQJ YHFWRUV FRQVWUXFWHG IURP WKH GDWD ^\Qf` ZKHUH / 1 fÂ§ 0O ,Q ZKDW IROORZV \=f LV UHIHUUHG WR DV WKH IRUZDUG GDWD YHFWRU /HW HZ=f O /fÂ§ EH IRUPHG IURP ^HZQf` LQ WKH VDPH PDQQHU DV \=f DUH IURP ^\Qf` 7KHQ WKH IRUZDUG YHFWRUV FDQ EH ZULWWHQ DV \ :X10MH Hf f ZKHUH LV WKH VWHHULQJ YHFWRU DQG LV JLYHQ E\ D PAf fÂ§ 7 H-8 f /LNHZLVH WKH EDFNZDUG GDWD YHFWRUV DUH FRQVWUXFWHG DV P \r1O f \r1O f \r^1O0f = / fÂ§ f ZKHUH ffr GHQRWHV WKH FRPSOH[ FRQMXJDWH /HW D-=f = / fÂ§ EH IRUPHG IURP ^HZQf` WKH VDPH ZD\ DV \=f IURP ^\Qf` 7KHQ WKH EDFNZDUG YHFWRU FDQ EH ZULWWHQ DV \=f >WWZfD0Zf@HnZL HD=f f ZKHUH mDf DrDfHBA-9fA f ,W LV VWUDLJKWIRUZDUG WR YHULI\ WKDW WKH IRUZDUG DQG EDFNZDUG YHFWRUV DUH UHODWHG E\ WKH IROORZLQJ FRPSOH[ FRQMXJDWH V\PPHWU\ SURSHUW\ \]f -\r7 fÂ§ ] fÂ§ Of f ZKHUH GHQRWHV WKH H[FKDQJH PDWUL[ ZKRVH DQWLGLDJRQDO HOHPHQWV DUH RQHV DQG DOO WKH RWKHUV DUH ]HUR 6XSSRVH WKDW WKH LQLWLDO SKDVH RI WKH VLQXVRLGDO VLJQDO LQ f LV D UDQGRP YDULDEOH XQLIRUPO\ GLVWULEXWHG RYHU WKH LQWHUYDO > WYf DQG LQGHSHQGHQW RI WKH QRLVH PAGE 30 WHUP %\ PDNLQJ XVH RI WKLV DVVXPSWLRQ DV ZHOO DV f WKH FRYDULDQFH PDWUL[ RI \Of RU HTXLYDOHQWO\ RI \Of LV JLYHQ E\ 5 ( ^\Of\+Of` ( ^\=f\+` Om0_ D0ZfDAZf 4Zf f ZKHUH f+ GHQRWHV WKH FRQMXJDWH WUDQVSRVH DQG 4FXf LV WKH QRLVH FRYDULDQFH PDWUL[ DQG LV JLYHQ E\ 40 %^0m` %^ifi` f 1RWH WKDW ERWK 5 DQG 4 DUH +HUPLWLDQ 7RHSOLW] PDWULFHV 7KH )RUZDUG2QO\ )2f LQWHUEDQN DSSURDFKHV XVH RQO\ WKH IRUZDUG GDWD YHFn WRU WR HVWLPDWH 5 ZKLFK LV WKH IRUZDUG VDPSOH FRYDULDQFH PDWUL[ / 5IR \ \\+ f A 7KH )RUZDUG%DFNZDUG )%f ILOWHUEDQN DSSURDFKHV XVH WKH DYHUDJH RI WKH IRUZDUG DQG EDFNZDUG VDPSOH FRYDULDQFH PDWULFHV WR REWDLQ WKH HVWLPDWH RI 5 5IE A5IR 5ERfL f ZKHUH 5ER GHQRWHV WKH EDFNZDUG VDPSOH FRYDULDQFH PDWUL[ /B 5%2 (n:\f MIW)2f A 7KH 5IE LQ f LV +HUPLWLDQ EXW QR ORQJHU 7RHSOLW] %\ XVLQJ f RQH FDQ VKRZ WKDW 5IE LV D SHUV\PPHWULF PDWUL[ >@ LH 5)E -5IEM f 6LQFH 5 LV SHUV\PPHWULF RQH ZRXOG H[SHFW WKDW 5IE LV D EHWWHU HVWLPDWH RI 5 WKDQ WKH QRQSHUV\PPHWULF 5SR PAGE 31 0$), )LOWHUV %\ GHILQLWLRQ WKH PDWFKHG ILOWHU LV GHVLJQHG VXFK WKDW WKH FRUUHVSRQGLQJ VLJQDOWRQRLVH 615f UDWLR LQ WKH ILOWHU RXWSXW LV PD[LPL]HG WKDW LV ?KbD0Xf? DUJ PD[ WU K40KZ Â‘ f 7KH VROXWLRQ LV REWDLQHG E\ PDNLQJ XVH RI WKH &DXFK\6FKZDUW] LQHTXDOLW\ VHH HJ >@f B 4}D P0 DeX!f4LXfDfZff ZKHUH 4X!f LV DVVXPHG WR EH LQYHUWLEOH ,W LV UHDGLO\ FKHFNHG WKDW WKH VROXWLRQ LQ f VDWLVILHV KD00 O f ZKLFK LPSOLHV WKDW WKH ILOWHU JLYHQ LQ f SDVVHV WKH IUHTXHQF\ XR XQGLVWRUWHG %\ PDNLQJ XVH RI WKLV REVHUYDWLRQ DQG RI f DQG f ZH KDYH K\=f DZfH}nZL KALZ / f DQG K"\ HrOIrfDn^X!fHfXO O / f 7KH OHDVW VTXDUHV /6f HVWLPDWH RI mDf REWDLQHG E\ XVLQJ RQO\ f LH WKH IRUZDUG GDWD YHFWRUV LV JLYHQ E\ f)R0 f ZKHUHDV WKH OHDVW VTXDUHV /6f HVWLPDWH RI DXf REWDLQHG E\ XVLQJ f DQG f WKDW LV ERWK WKH IRUZDUG DQG EDFNZDUG GDWD YHFWRUV LV JLYHQ E\ mIE0 A >KAIJZf HaÂƒ1fX-J+Xf.@ f ZKHUH JZf DQG JZf DUH UHVSHFWLYHO\ WKH QRUPDOL]HG )RXULHU WUDQVIRUPV RI WKH IRUZDUG DQG EDFNZDUG GDWD YHFWRUV L0 / f PAGE 32 /B JZf =< \OfHa-8O f A Â R 6LQFH 4D!f LV 7RHSOLW] DQG WKHUHIRUH SHUV\PPHWULF RQH FDQ VKRZ WKDW WKH KZ LQ f VDWLVILHV >@ KZHM0fZ f &RQVHTXHQWO\ LW IROORZV DIWHU VRPH FDOFXODWLRQ WKDW f LV HTXLYDOHQW WR t)%Xf KrJXf f +HQFH GXH WR WKH SHUV\PPHWU\ RI 4X!f WKH )% HVWLPDWH RI D^Xf KDV WKH VDPH IRUP DV WKH )2 HVWLPDWH RI DX!f VHH f ff +RZHYHU WKH ILOWHU YHFWRU KZ REWDLQHG ZLWK WKH )2 DSSURDFK LV LQ JHQHUDO GLIIHUHQW IURP WKDW FRUUHVSRQGLQJ WR WKH )% DSSURDFK >@ $OWKRXJK QHLWKHU &DSRQ QRU $3(6 ZDV GHULYHG LQ WKH 0$), IUDPHZRUN IRU RULJLQDO GHULYDWLRQV RI WKHVH PHWKRGV ZH UHIHU WR >@ >@ >@f LQ ZKDW IROORZV ZH VKRZ WKDW WZR QDWXUDO HVWLPDWRUV RI 4Zf LQ f OHDG WR WKH &DSRQ DQG $3(6 ILOWHUV UHVSHFWLYHO\ 0RUH LQWHUHVWLQJO\ ZH DOVR VKRZ WKDW HYHQ WKRXJK D WKLUG QDWXUDO HVWLPDWRU RI 4FMf JLYHV D QHZ ILOWHU ZKLFK LV GLIIHUHQW IURP WKH IRUPHU WZR WKH VSHFWUDO HVWLPDWRU FRUUHVSRQGLQJ WR WKH QHZ ILOWHU WXUQV RXW WR EH HTXLYDOHQW WR $3(6 DV ZHOO &DSRQ )LOWHU %\ f RQH QDWXUDO FKRLFH LV WR HVWLPDWH 4Xf DV 4&DSRQ0 5 a _Â£Zf_ D0ZfDeMf f ZKHUH Â£Zf GHQRWHV DQ HVWLPDWH RI TcRMf DQG 5 GHQRWHV HLWKHU 5SR RU 5)%L ZKLFK LQ WXUQ OHDGV WR )2& RU )%& UHVSHFWLYHO\ )RU QRWDWLRQDO VLPSOLFLW\ LQ ZKDW IROORZV WKH IRUZDUGRQO\ &DSRQ $3(6 DQG 0$), VSHFWUDO HVWLPDWRUV ZLOO EH UHIHUUHG WR DV )2& )2$ DQG )20 UHVSHFWLYHO\ /LNHZLVH )%& )%$ DQG )%0 UHSUHVHQW WKH FRUUHVSRQGLQJ IRUZDUGEDFNZDUG FRXQWHUSDUW HVWLPDWRUVf %\ PDNLQJ XVH RI WKH PAGE 33 PDWUL[ LQYHUVLRQ OHPPD >@ RQH FDQ VHH WKDW WKH VHFRQG WHUP LQ f KDV QR LQIOXHQFH RQ WKH KZ LQ f +HQFH ZKHQ f LV VXEVWLWXWHG LQWR f WKH PDWFKHG ILOWHU UHGXFHV WR WKH &DSRQ ILOWHU >@ >@ 5 DMAXf5D0Df f 2EVHUYH WKDW 4WXf LV SHUV\PPHWULF IRU HLWKHU )2& RU )%& %\ VXEVWLWXWLQJ f LQWR f RU f ZH REWDLQ WKH &DSRQ HVWLPDWH RI $3(6 )LOWHU A&DSRQ&Af DMARf5AZf f ,JQRULQJ WKH IDFW WKDW DPORf LV NQRZQ ZH REWDLQ WKH /6 HVWLPDWH RI WKH YHFWRU DZfD0Zf LQ f DV >DXfD0XMf@ JX!f f ,QVHUWLQJ f LQWR f DORQJ ZLWK 5SR VXEVWLWXWHG \LHOGV WKH )2$ HVWLPDWH RI 40 4I2D0 5)2 JZfJ+Zf f $ SHUV\PPHWULF HVWLPDWH RI 4Xf FDQ EH REWDLQHG E\ XVLQJ ERWK WKH IRUZDUG DQG EDFNZDUG GDWD YHFWRUV 4IED0 A >4I2D0 -4I2DZf5)% *Zf*LXf f ZKHUH *Zf 9 JFXf JXf f DQG ZH KDYH XVHG WKH IDFW WKDW -JrZf HA/aAJRMf +HQFH E\ f ZH REWDLQ WKH WKH $3(6 ILOWHU >@ $3(6 B 4AOAfD0Zf K DZZf4 0D0Zf f PAGE 34 ZKHUH 4Zf GHQRWHV HLWKHU 4ITD RU 4IE$ ZKLFK FRUUHVSRQGV WR )2$ RU )%$ &RQVHTXHQWO\ WKH $3(6 HVWLPDWH RI DXf LV JLYHQ E\ VHH f DQG ff D0Zf4ZfJZf mDSHV0 f $QRWKHU 0DWFKHG )LOWHU (TXDWLRQV f DQG f VXJJHVW DQRWKHU ZD\ WR HVWLPDWH WKH )2 DQG )% HVWLn PDWH RI 4Zf LQ ZKDW IROORZV ZH VRPHWLPHV RPLW WKH GHSHQGHQFH RQ X! IRU QRWDWLRQDO FRQYHQLHQFHf 4IRP0 ?OaL M PAGE 35 DQG Â3%D0 S fÂ§ Â‘A)% r n 9J DJM DA5SJ 5SJ DrJ aa RD0 +RO UUW)% GrJ DJM D0 5SJ D0 D0 A)% DrJ DJ 5S%D0 Dn$LnAn)%D0nAn)% DrJ DJ fÂ§D+ 5 G$LU/)% DrJ DJ f ZKLFK JLYHV WKH IROORZLQJ H[SUHVVLRQV IRU WKH PDWFKHG ILOWHU YHFWRUV AS4D0 DQG )20 D0A )2DZ 5S2D$L prD0An)2D0nAn)26 fÂ§ DrD0A)&f6nAn)2D0 DA5S4D0 f )%0 A3%D$L D:A)%D0 5S%D0 D0An)%D0nAn)% Â£rJ p J M&5 + 'O 0 )% trJ D J 5n)%D$L D0A)%D0 f 7KH SUHYLRXV 0$), ILOWHUV DUH LQ JHQHUDO GLIIHUHQW IURP ERWK WKH &DSRQ DQG $3(6 ILOWHUV VLQFH QHLWKHU WKH &DSRQ QRU WKH $3(6 ILOWHUV GHSHQG RQ DQ HVWLPDWH RI DRMf ZKLFK WKH QHZ 0$), ILOWHUV QHHG WR NQRZ ,Q VSLWH RI WKLV IDFW LQ $SSHQGL[ $ ZH SURYH WKDW IRU D FHUWDLQ QDWXUDO FKRLFH RI DX!f LQ f DQG f WKH IROORZLQJ HTXDOLWLHV KROG WUXH m)2P0 m72D0 f fIEP0 mIED0 f PAGE 36 $QDO\VHV RI 0$), $SSURDFKHV &RPSXWDWLRQDO &RPSOH[LW\ /HW 5S4 DQG 5SJ GHQRWH WKH +HUPLWLDQ VTXDUH URRWV RI WKH SRVLWLYH GHILQLWH PDWULFHV 5S4 DQG 5SJ UHVSHFWLYHO\ 'HILQH LLXf 5S4 DZIRf f 0Zf 5SR f 7KHQ )2& DQG )2$ FDQ EH H[SUHVVHG DV UHODWLYHO\ VLPSOH IXQFWLRQV RI AZf DQG ]Xf VHH f DQG ff f)2F0 ,,DO+,, f DQG fIRD0 0O KPP ,,PL0,, ,,PL+,, ,,P0__ ,P"ZfP0,f $SSO\LQJ WKH PDWUL[ LQYHUVLRQ OHPPD WR f \LHOGV L L 4)%$0 fÂ§ A)% b%*Zf Zf5S%*N2 A *IIMf5SJ f f f ZKHUH LV WKH [ LGHQWLW\ PDWUL[ 1H[W GHILQH mnO0 5SJD0Zf YXf 5SMIJ+ XXf 5SJJZf 7KHQ WKH )%& DQG )%$ VSHFWUDO HVWLPDWRUV FDQ EH H[SUHVVHG DV VHH f DQG ff fIEF0 8LX`fX^XMf ,NL+,, f f f f f PAGE 37 DQG XJXMfYXf fIED0 ,,AL+,, ? .ZfA+ I ZfLZf@ 6 AZf f ZKHUH f +HQFH FRPSXWDWLRQDOO\ $3(6 HVSHFLDOO\ )2$ LV RQO\ VOLJKWO\ PRUH LQYROYHG WKDQ &DSRQ $OVR VHH 6HFWLRQ IRU WKH VLPXODWLRQ UHVXOWVf 0RUH VSHFLILFDOO\ WKH DPRXQW RI FRPSXWDWLRQV UHTXLUHG E\ &DSRQ RU $3(6 LV GRPLQDWHG E\ FDOFXODWLQJ 5 DQG WKH PDWUL[YHFWRU SURGXFWV LQ ff RU ff :H PHQWLRQ WKDW FRQYHQWLRQDO &DSRQ RU $3(6 LPSOHPHQWDWLRQ PDNHV XVH RI f ff f RU ff ZKLFK UHTXLUHV FDOFXODWLQJ WKH PDWUL[YHFWRU SURGXFWV LQ f f RU ff IRU HDFK Z RI LQWHUHVW WKHUHE\ EHFRPLQJ FRPSXWDWLRQDOO\ PRUH DQG PRUH LQWHQVLYH DV WKH QXPEHU RI IUHTXHQF\ VDPSOHV LQFUHDVHV 7KLV LV HVSHFLDOO\ VR LQ DSSOLFDWLRQV VXFK DV ZKHQ IRUPLQJ 6$5 LPDJHV ,W LV WKXV RI JUHDW LQWHUHVW LI RWKHU HIILFLHQW LPSOHPHQWDWLRQ VFKHPHV IRU WKH 0$), DSSURDFKHV FDQ EH IRXQG 6WDWLVWLFDO 3HUIRUPDQFH $OO HVWLPDWRUV XQGHU VWXG\ LH )2& )2$ )%& DQG )%$ FDQ EH VKRZQ WR KDYH WKH VDPH DV\PSWRWLF YDULDQFH XQGHU WKH IROORZLQJ FRQGLWLRQ & 7KH VLJQDO \Qf FDQ EH ZULWWHQ DV LQ f ZKHUH HZQf LV D ]HURPHDQ VWDWLRQDU\ UDQGRP SURFHVV ZLWK ILQLWH VSHFWUDO GHQVLW\ DW X !H^Xf 22 f ,Q PRUH H[DFW WHUPV WKH IROORZLQJ UHVXOW KROGV WUXH PAGE 38 7KHRUHP 8QGHU &RQGLWLRQ & DQG WKH DGGLWLRQDO DVVXPSWLRQ WKDW HZQf LV FLUFXODUO\ V\PPHWULFDOO\ GLVWULEXWHG WKH HVWLPDWLRQ HUURUV LQ WKH &DSRQ DQG $3(6 VSHFWUDO HVWLPDWRUV DUH DV\PSWRWLFDOO\ FLUFXODUO\ V\PPHWULFDOO\ GLVWULEXWHG ZLWK ]HUR PHDQ DQG WKH IROORZLQJ FRPPRQ YDULDQFH OLP /( _Â£FMf fÂ§ DRf_` f 3URRI 6HH $SSHQGL[ % 7KH QHHG WR HQIRUFH &RQGLWLRQ & OLPLWV WR VRPH H[WHQW WKH LPSRUWDQFH RI WKH SUHYLRXV UHVXOW ,QGHHG WKH DVVXPSWLRQ PDGH LQ & LV VDWLVILHG LI DQG HVVHQWLDOO\ RQO\ LIf WKH VLJQDO \Qf KDV D PL[HG VSHFWUXP DQG XM LV WKH ORFDWLRQ RI D VSHFWUDO OLQH 7KH UHVXOW RI 7KHRUHP LV UHOHYDQW WR WKH VSHFWUDO DQDO\VLV RI D WDUJHW ZLWK GRPLQDQW SRLQW VFDWWHUHUV LQ WKH SUHVHQFH RI GLVWULEXWHG FOXWWHU VHH >@ DQG WKH UHIHUHQFHV WKHUHLQf ,Q VRPH RWKHU DSSOLFDWLRQV KRZHYHU WKH PDLQ LQWHUHVW LV LQ WKH FRQWLQXRXV FRPSRQHQW RI WKH VSHFWUXP )RU H[DPSOH &RQGLWLRQ & GRHV QRW KROG H[DFWO\ IRU D WDUJHW ZLWK GLVWULEXWHG VFDWWHUHUV VLQFH WKH VLJQDWXUH VSHFWUXP LV FRQWLQXRXV DW X! 7KDW WKH SUHYLRXV UHVXOW LV RI D VRPHZKDW OLPLWHG LQWHUHVW LV DOVR GXH WR LWV DV\PSWRWLF FKDUDFWHU ,QGHHG LQ DSSOLFDWLRQV ZLWK PHGLXP RU VPDOOVL]HG GDWD VDPn SOHV WKH VSHFWUDO HVWLPDWRUV XQGHU VWXG\ KDYH EHHQ IRXQG WR EHKDYH TXLWH GLIIHUHQWO\ LQ FRQWUDGLFWLRQ ZLWK ZKDW LV SUHGLFWHG E\ WKH DV\PSWRWLFf UHVXOW RI 7KHRUHP VHH WKH QXPHULFDO H[DPSOHV LQ 6HFWLRQ f 7KH ILQLWHVDPSOH DQDO\VLV RI WKH VSHFWUDO HVn WLPDWRUV XQGHU GLVFXVVLRQ ZRXOG FRQVHTXHQWO\ EH RI FRQVLGHUDEOH LQWHUHVW +RZHYHU D FRPSOHWH DQDO\VLV LI SRVVLEOH DSSHDUV WR EH UDWKHU GLIILFXOW DW EHVW $ SDUWLDO RQH E\ PDNLQJ XVH RI D KLJKHURUGHU 7D\ORU H[SDQVLRQ WHFKQLTXH LV QHYHUWKHOHVV IHDVLEOH 7KH UHVXOW LV DV IROORZV 7KHRUHP 7R ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQ DQG XQGHU WKH PLOG DVn VXPSWLRQ WKDW WKH WKLUGRUGHU PRPHQWV RI HZQf DQG HZQf DUH ]HUR WKH &DSRQ DQG PAGE 39 $3(6 HVWLPDWRUV DUH UHODWHG E\ /( ^Â£)%&Xf DXMf` /(^D)&Xf DZf` ? 7 f D>LRf 2LLMf DQG /( ^$)ED0 m0, (( ^mIRDZf a DZf` f IRU VXIILFLHQWO\ ODUJH YDOXHV RI / 3URRI 6HH $SSHQGL[ & :H EHOLHYH WKDW f DQG f SURYLGH D WKHRUHWLFDO PRWLYDWLRQ IRU SUHIHUn ULQJ $3(6 WR &DSRQ LQ PRVW VSHFWUDO HVWLPDWLRQ H[HUFLVHV 0RUHRYHU 7KHRUHP DOVR VXJJHVWV WKDW )%& VKRXOG EH SUHIHUUHG RYHU )2& :KLOH ERWK )2$ DQG )%$ DUH VLPLODUO\ XQELDVHG ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQf WKH ODWWHU LV XVXDOO\ REVHUYHG ZLWK VOLJKWO\ EHWWHU UHVROXWLRQ DQG VLGHOREH SURSHUWLHV >@ DW WKH FRVW RI VOLJKWO\ PRUH FRPSXWDWLRQV ([WHQVLRQV :H EULHIO\ GHVFULEH WKH H[WHQVLRQV RI WKH 0$), VSHFWUDO HVWLPDWRUV :H ILUVW GHFRPSRVH WKH REVHUYDWLRQV ^\QLQf` DV WQLQf RDL:fHMf-LQLDQfH:OL:QQf f Q[ : fÂ§ Q 1 fÂ§ XT FM ( > WWf ZKHUH DX!LXf GHQRWHV WKH FRPSOH[ DPSOLWXGH RI D VLQXVRLGDO VLJQDO ZLWK IUHn TXHQF\ X!LX!f DQG H8OL8QL Qf GHQRWHV WKH QRLVH RU UHVLGXDOf WHUP DW IUHTXHQF\ XTXf DVVXPHG WR EH ]HURPHDQ 1H[W LQ D PDQQHU VLPLODU WR WKH FDVH ZH IRUP WKH 0L [ 0 IRUZDUG DQG EDFNZDUG GDWD PDWULFHV <ÂXÂf ^\Q?Qf Q? =L O? 0? fÂ§ Q fÂ§ = O 0 fÂ§ ` <=L=f ^\rQLQf Q[ 1LOLO1LOL0L@ Q 1 fÂ§ O fÂ§ 1 fÂ§ O fÂ§ 0` K /L fÂ§ O / fÂ§ f PAGE 40 ZKHUH /? 1L fÂ§ 0L DQG / fÂ§ 1 fÂ§ 0 /HW \Lf YHF>< KOf? f \KOf YHF> PAGE 41 RQH FDQ VHH WKDW 5 LV SHUV\PPHWULF 6LPLODUO\ 4 LV DOVR SHUV\PPHWULF 7KH IRUZDUGEDFNZDUG VDPSOH FRYDULDQFH PDWUL[ WDNHV WKH IRUP U)% 5)2 U%f! f ZKHUH 5SR DQG 5ER GHQRWH WKH VDPSOH FRYDULDQFH PDWULFHV RI ^\Â Âf` DQG ^\K =f` UHVSHFWLYHO\ JLYHQ E\ 5SR 5%2 /?/ /? fÂ§ ? /a < < \KKf\+^KLf K /L/ ((:X:GXNf OO c f f %\ PDNLQJ XVH RI f RQH FDQ VHH WKDW 5IE LV DOVR SHUV\PPHWULF /HW +:O$ GHQRWH WKH LPSXOVH UHVSRQVH RI DQ [ 0 ),5 ILOWHU DQG OHW KRL YHFI+AMA@ f /LNH LQ WKH FDVH WKH LPSXOVH UHVSRQVH RI WKH PDWFKHG ILOWHU LV JLYHQ E\ B 4D-LDcfD0L0ZL! Zf ;:OK D0L P ZL f Zf4 DcL RcfD0O P ZL Af f 1RWH WKDW KZL 8D0L0ZL 9f fÂ§ f 7KH /6 HVWLPDWHV RI X!f REWDLQHG E\ XVLQJ RQO\ WKH IRUZDUG GDWD YHFWRUV DQG E\ XVLQJ ERWK WKH IRUZDUG DQG EDFNZDUG GDWD YHFWRUV DUH JLYHQ E\ VLPLODUO\ WR f DQG f LQ WKH FDVHf Â£)RZLDcf KJZf f DQG M1LOfXcLHM1 f PAGE 42 ZKHUH JZLZf Â \^KKfH -nZL:LDf L R JZLZf / fÂ§ = fÂ§ /?/Â \L!fH MZLÂL:f f f R Â R 6LQFH 4 LV SHUV\PPHWULF f FDQ EH ZULWWHQ DV p)EZL! Af fÂ§ KOL:JZL Zf 7KH &DSRQ PHWKRG HVWLPDWHV WKH QRLVH FRYDULDQFH PDWUL[ DV f 4FDSRQALfAf fÂ§ 5 _DZL:f_D$Lf0AAfDAOL0DLZf f ZKHUH ÂccFM[Af GHQRWHV VRPH HVWLPDWH RI JFMLMf DQG 5 GHQRWHV HLWKHU 5SR RU 5SJ ZKLFK FRUUHVSRQG WR )2& RU )%& 7KXV WKH &DSRQ HVWLPDWH RI DX!LFMf LV REWDLQHG DV A&DSRQ ZOf Af D0X0XL9f5 ;JZLWAf D0L0:OL-f5 :f 7KH )2 DQG )% $3(6 HVWLPDWHV RI 4X!LDf WDNH WKH IRUP 4I2$:f:f 5I2 fÂ§ JAOAfJAfOfAOf f DQG ZKHUH 4IEDZL Af fÂ§ 5IE fÂ§ *ZL FMf*KFML XMLf *FMLFMf fÂ§ \ JX-L:f JLGL8!f f f f +HQFH ZH REWDLQ Â$3(6 AOf Af DV D00:O!Df4 :OAfJAO:f Â£$3(VZLfZf D0L 0 Z Zf4 A AfD0L 0 WAO Af ZKHUH 4XfLFMf GHQRWHV HLWKHU 4I2DZL!FMf RU 4IEDZL! Af f PAGE 43 %DVHG RQ WKH H[WHQVLRQV GHVFULEHG DERYH LW LV QRW GLIILFXOW WR VHH WKDW DOO WKH UHVXOWV RI WKH SUHYLRXV VHFWLRQ DOVR KROG WUXH IRU WKH &DSRQ DQG $3(6 HVWLPDWRUV ,QGHHG WKH SURRIV IRU WKH HVWLPDWRUV IROORZ D VLPLODU SDWWHUQ WR WKRVH IRU WKH FDVH VKRZQ LQ $SSHQGL[HV % DQG & 1XPHULFDO ([DPSOHV ,Q WKH IROORZLQJ ZH VWXG\ WKH &DSRQ DQG $3(6 FRPSOH[ DPSOLWXGH HVWLPDWHV LQ D QXPEHU RI FDVHV RI LQWHUHVW )RU ERWK WKH DQG H[DPSOHV JLYHQ EHORZ ZH FRPSDUH WKH SHUIRUPDQFH RI WKH IRUZDUGRQO\ &DSRQ DQG $3(6 DV ZHOO DV WKH IRUZDUGEDFNZDUG &DSRQ DQG $3(6 ZKLFK DUH IRU VLPSOLFLW\ UHIHUUHG WR DV )&DSRQ )$3(6 )%&DSRQ DQG )%$3(6 UHVSHFWLYHO\ &RPSOH[ 6SHFWUDO (VWLPDWLRQ 7KH GDWD XVHG LQ WKH H[DPSOHV FRQVLVWV RI D VXP RI FRPSOH[ VLQXVRLGV ZLWK WKH UHDO DQG LPDJLQDU\ SDUWV VKRZQ LQ )LJXUHV Df DQG Ef UHVSHFWLYHO\ FRUUXSWHG E\ D ]HURPHDQ FRPSOH[ ZKLWH *DXVVLDQ QRLVH 7KH GDWD OHQJWK LV FKRVHQ DV 1 ,Q ZKDW IROORZV ZH DUH LQWHUHVWHG LQ WKH ELDV DQG YDULDQFH SURSHUWLHV RI WKH HVWLPDWRUV XQGHU VWXG\ 7KH ELDV DQG YDULDQFH UHVXOWV VKRZQ EHORZ FRUUHVSRQG WR WKH IUHTXHQF\ RI WKH ILUVW VLQXVRLG DQG WKH\ DUH REWDLQHG IURP LQGHSHQGHQW UHDOL]DWLRQV :H EHJLQ E\ VWXG\LQJ WKH SHUIRUPDQFH RI WKH HVWLPDWRUV DV WKH VLJQDOWRQRLVH UDWLR 615f YDULHV 7KH 615 IRU WKH NWK VLQXVRLG LV GHILQHG DV 615IH RJAK G%f f HAFf ZKHUH LV WKH FRPSOH[ DPSOLWXGH RI WKH NWK VLQXVRLG DQG 3H^XfNf LV WKH VSHFWUDO GHQVLW\ RI WKH DGGLWLYH QRLVH DW IUHTXHQF\ 7KH ILOWHU OHQJWK LV FKRVHQ DV 0 7KH UHDO DQG LPDJLQDU\ SDUWV RI WKH ELDV DUH VKRZQ LQ )LJXUHV Df DQG Ef UHVSHFWLYHO\ DV D IXQFWLRQ RI 615L $V VHHQ IURP WKHVH ILJXUHV )2$ DQG )%$ DUH PAGE 44 DOPRVW XQELDVHG ZKLOH )2& DQG )%& DUH ELDVHG GRZQZDUG ,Q DGGLWLRQ ZH QRWLFH WKDW WKH ELDV IRU )2& LV DSSUR[LPDWHO\ WZLFH WKDW RI )%& $OO WKHVH REVHUYDWLRQV DUH FRQVLVWHQW ZLWK WKH SUHGLFWLRQ RI WKH WKHRU\ 7KH YDULDQFHV RI WKH UHDO DQG LPDJLQDU\ SDUWV RI WKH DPSOLWXGH HVWLPDWHV DUH VKRZQ LQ )LJXUHV Ff DQG Gf UHVSHFWLYHO\ ,W DSSHDUV WKDW DOO RI WKH HVWLPDWRUV GLVSOD\ VLPLODU YDULDQFHV +RZHYHU DV VKRZQ LQ WKH QH[W H[DPSOH WKH YDULDQFH RI &DSRQ EHFRPHV QRWDEO\ ODUJHU WKDQ WKDW RI $3(6 DV 0 LQFUHDVHV 1H[W ZH VWXG\ WKH HIIHFW RI WKH ILOWHU OHQJWK 0 RQ WKH HVWLPDWRUV 7KH 615L LV IL[HG DW G% $V 0 YDULHV WKH UHDO DQG LPDJLQDU\ SDUWV RI WKH ELDV DUH VKRZQ LQ )LJXUHV Df DQG Ef UHVSHFWLYHO\ )URP WKHVH ILJXUHV RQH FDQ VHH WKDW ERWK )2$ DQG )%$ DUH XQELDVHG IRU DOO SUDFWLFDO ILOWHU OHQJWKV ZKHUHDV WKH ELDV RI &DSRQ JURZV VLJQLILFDQWO\ ZLWK LQFUHDVLQJ 0 $ SUDFWLFDO ILOWHU OHQJWK PHDQV WKDW 0 VKRXOG QRW EH WRR VPDOO >@ ,Q IDFW DOO ILOWHUEDQN PHWKRGV UHGXFH WR WKH )RXULHU WUDQVIRUP DSSURDFK ZKHQ 0 fÂ§ DQG RQO\ ZKHQ 0 LV VXIILFLHQWO\ ODUJH WKH ILOWHUEDQN DSSURDFK VKRZV QRWLFHDEOH LPSURYHPHQW RYHU WKH )RXULHU PHWKRG >@f $OO HVWLPDWRUV VHHP WR SHUIRUP VLPLODUO\ IRU 0 XS WR D IRXUWK RI WKH GDWD OHQJWK ZLWK &DSRQ EHLQJ VOLJKWO\ ELDVHG GRZQZDUG $V WKH ILOWHU OHQJWK LQFUHDVHV IXUWKHU WKH SHUIRUPDQFH RI &DSRQ GHJUDGHV UDSLGO\ ZKLOH WKDW RI $3(6 UHPDLQV XQDIIHFWHG 7KLV REVHUYDWLRQ LV VWUHQJWKHQHG E\ WKH YDULDQFH UHVXOWV VKRZQ IRU WKH UHDO DQG LPDJLQDU\ SDUWV RI WKH DPSOLWXGH HVWLPDWHV LQ )LJXUHV Ff DQG Gf UHVSHFWLYHO\ ,W LV NQRZQ WKDW DV 0 LQFUHDVHV DOO RI WKH HVWLPDWRUV XQGHU VWXG\ DFKLHYH EHWWHU VSHFWUDO UHVROXWLRQ DQG WKDW WKH EHVW UHVROXWLRQ LV REWDLQHG DW 0 1 >@ 7KLV IDFW DORQJ ZLWK WKH VWDWLVWLFDO UHVXOWV VKRZQ LQ WKH SUHYLRXV H[DPSOHV LQGLFDWHV WKDW WKH FKRLFH RI 0 IRU &DSRQ VKRXOG EH PDGH E\ D WUDGHRII EHWZHHQ UHVROXWLRQ DQG VWDWLVWLFDO VWDELOLW\ 8VXDOO\ ZH FKRRVH 1 0 1 :KLOH WKH FKRLFH RI 0 IRU &DSRQ LV GLIILFXOW WR PDNH LW LV HDV\ WR VHH WKDW $3(6 DFKLHYHV WKH EHVW SHUIRUPDQFH DW 0 1 VLQFH ZLWK WKLV FKRLFH $3(6 DFKLHYHV WKH KLJKHVW SRVVLEOH PAGE 45 ' &RPSOH[ 6LQXVRLGV &RPSOH[ 6LQXVRLGV Df Ef )LJXUH 7KH FRPSOH[ DPSOLWXGH RI WKH VXP RI VLQXVRLGV XVHG LQ WKH VLPXODWLRQV Df 5HDO SDUW Ef ,PDJLQDU\ SDUW UHVROXWLRQ DV ZHOO DV WKH EHVW VWDWLVWLFDO SURSHUWLHV LQ WHUPV RI ELDV DQG YDULDQFH 7KH SUHYLRXV H[DPSOHV DOVR VKRZ WKDW )2$ DQG )%$ SHUIRUP VLPLODUO\ LQ WHUPV RI ELDV DQG YDULDQFH SURSHUWLHV IRU WKH IUHTXHQF\ RI LQWHUHVW 7R FRPSDUH WKH FRPSXWDWLRQDO FRPSOH[LWLHV RI WKH HVWLPDWRUV XQGHU VWXG\ ZH FRXQW WKH IORSV UHTXLUHG E\ HDFK RI WKHP IRU WKH FDVH ZKHUH 1 0 DQG WKH FRPSOH[ VSHFWUD DUH HYDOXDWHG DW HTXDOO\ VSDFHG SRLQWV 7KH IORSV UHTXLUHG E\ )2& DQG )%& DUH DSSUR[LPDWHO\ WKH VDPH ZKHUHDV WKH IORSV QHHGHG E\ )2$ DQG )%$ DUH UHVSHFWLYHO\ DQG WLPHV RI WKDW E\ WKH &DSRQ HVWLPDWRUV &RPSOH[ 6SHFWUDO (VWLPDWLRQ $V ZDV PHQWLRQHG LQ 6HFWLRQ WKH &DSRQ DQG $3(6 HVWLPDWRUV EHn KDYH UDWKHU VLPLODUO\ WR WKHLU FRXQWHUSDUWV 6LQFH WKH SUREOHPV HQFRXQWHUHG LQ DSSOLFDWLRQV VXFK DV V\QWKHWLF DSHUWXUH UDGDU LPDJLQJ DUH FRQFHUQHG ZLWK FRPn SOH[ VSHFWUDO HVWLPDWLRQ ZH LQFOXGH D FRXSOH RI QXPHULFDO H[DPSOHV KHUH 7KH GDWD HPSOR\HG FRQVLVWV RI WKUHH VLQXVRLGV FRUUXSWHG E\ D ]HURPHDQ FRPSOH[ ZKLWH *DXVVLDQ QRLVH ZLWK 1? 1 7KH VLQXVRLGV DUH ORFDWHG LQ WKH IUHTXHQF\ GRPDLQ DW f f DQG f DQG WKHLU DPSOLWXGHV DUH H-Un H-U PAGE 46 Df Ef Ff Gf )LJXUH (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV 615M YDULHV ZKHQ 1 DQG 0 Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH PAGE 47 ' &RPSOH[ $PSOLWXGH (VWLPDWLRQ &RPSOH[ $PSOLWXGH (VWLPDWLRQ &RPSOH[ $PSOLWXGH (VWLPDWLRQ &RPSOH[ $PSOLWXGH (VWLPDWLRQ &f Gf )LJXUH (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV WKH ILOWHU OHQJWK 0 YDULHV ZKHQ 1 DQG 615L G% Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH PAGE 48 DQG H-U UHVSHFWLYHO\ 7KH ELDV DQG YDULDQFH IRU WKH DPSOLWXGH HVWLPDWH RI WKH ILUVW VLQXVRLG DUH REWDLQHG IURP LQGHSHQGHQW UHDOL]DWLRQV 7KH 615 IRU WKH NWK VLQXVRLG LV VLPLODUO\ GHILQHG DV LQ f 7KH ELDV DQG YDULDQFH RI WKH IRXU HVWLPDWRUV XQGHU VWXG\ YHUVXV 615L DUH VKRZQ LQ )LJXUHV Df WR Gf UHVSHFn WLYHO\ ZKHUH 0L 0 )LJXUHV Df WR Gf VKRZ WKH VWDWLVWLFDO UHVXOWV DV WKH ),5 ILOWHU OHQJWK YDULHV ZKHUH 615L LV IL[HG DW G% :H DVVXPH LQ )LJXUHV Df WR Gf WKDW 0L 0 $V VHHQ IURP WKHVH SORWV WKH SHUIRUPDQFH RI WKH 0$), HVWLPDWRUV LQGHHG UHVHPEOHV WKDW RI WKHLU FRXQWHUSDUWV DQG WKHUHIRUH ZH UHIHU WKH UHDGHUV WR WKH H[DPSOHV IRU FRPPHQWV 6XPPDU\ 7KLV FKDSWHU GLVFXVVHV XVLQJ WKH 0$), DSSURDFK IRU FRPSOH[ VSHFWUDO HVWLPDn WLRQ 7KH &DSRQ DQG $3(6 HVWLPDWRUV DUH VKRZQ WR EH PHPEHUV RI WKH 0$), FODVV %\ XVLQJ D KLJKHURUGHU H[SDQVLRQ WHFKQLTXH LW LV SURYHG WKDW WR ZLWKLQ D VHFRQG RUGHU DSSUR[LPDWLRQ &DSRQ LV ELDVHG GRZQZDUGf ZKLOH $3(6 LV XQELDVHG DQG WKDW WKH ELDV RI WKH IRUZDUGEDFNZDUG &DSRQ LV RQH KDOI WKDW RI WKH IRUZDUGRQO\ &DSRQ ,W LV DOVR VKRZQ WKDW WKH DERYH FRQFOXVLRQV FDUU\ RYHU WR WKH 0$), HVWLPDWRUV DV ZHOO 6LQFH FRPSXWDWLRQDOO\ $3(6 LV RQO\ VOLJKWO\ PRUH LQYROYHG WKDQ &DSRQ WKH SUHIHUHQFH RI $3(6 WR &DSRQ LQ SUDFWLFDO DSSOLFDWLRQV IROORZV ORJLFDOO\ EHFDXVH RI WKH EHWWHU VWDWLVWLFDO SURSHUWLHV DVVRFLDWHG ZLWK WKH IRUPHU PAGE 49 ' &RPSOH[ $PSOLWXGH (VWLPDWLRQ &RPSOH[ $PSOLWXGH (VWLPDWLRQ Df Ef Ff Gf )LJXUH (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV WKH 615L YDULHV ZKHQ 1L 1 DQG 0L 0 Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH PAGE 50 ' &RPSOH[ $PSOLWXGH (VWLPDWLRQ &RPSOH[ $PSOLWXGH (VWLPDWLRQ Df Ef Ff Gf )LJXUH (PSLULFDO ELDV DQG YDULDQFH RI WKH 0$), HVWLPDWRUV DV WKH ILOWHU OHQJWK 0 0? 0 YDULHV ZKHQ 1? 1 DQG 615L G% Df 5HDO SDUW RI WKH ELDV Ef ,PDJLQDU\ SDUW RI WKH ELDV Ff 9DULDQFH RI WKH UHDO SDUW RI WKH HVWLPDWHG DPSOLWXGH Gf 9DULDQFH RI WKH LPDJLQDU\ SDUW RI WKH HVWLPDWHG DPSOLWXGH PAGE 51 &+$37(5 ()),&,(17 ,03/(0(17$7,21 2) &$321 $1' $3(6 ,QWURGXFWLRQ ,Q WKH SUHYLRXV FKDSWHU ZH UHGHULYHG WKH &DSRQ DQG $3(6 VSHFWUDO HVWLPDWRUV XVLQJ WKH 0$), DSSURDFK ,Q WKH 0$), IUDPHZRUN D QXPEHU RI VWDWLVWLFDO SURSHUWLHV RI &DSRQ DQG $3(6 ZHUH REWDLQHG +RZHYHU ZH GLG QRW DGGUHVV YHU\ FDUHIXOO\ WKH LPSOHPHQWDWLRQ RI WKH &DSRQ DQG $3(6 HVWLPDWRUV (TXDWLRQV f f f DQG f JLYH WKH LQWXLWLYH ZD\V RI LPSOHPHQWLQJ &DSRQ DQG $3(6 +RZHYHU VXFK LQWXLWLYH LPSOHPHQWDWLRQV DUH FRPSXWDWLRQDOO\ H[SHQVLYH HVSHFLDOO\ IRU VSHFWUDO HVWLPDWLRQ IURP GDWD VHTXHQFHV ,Q WKLV FKDSWHU ZH VWXG\ KRZ WR LPSOHPHQW &DSRQ DQG $3(6 HIILFLHQWO\ )RU VLPSOLFLW\fV VDNH ZH RQO\ FRQVLGHU VSHFWUDO HVWLPDWLRQ VLQFH VSHFWUDO HVWLPDWLRQ LV D VSHFLDO FDVH RI WKH IRUPHU ZH DOVR RQO\ FRQVLGHU IRUZDUGEDFNZDUG &DSRQ DQG $3(6 VLQFH WKH\ DUH PRUH RIWHQ XVHG WKDQ WKHLU IRUZDUGRQO\ FRXQWHUSDUWV (IILFLHQW ,PSOHPHQWDWLRQ RI $3(6 )LUVW ZH UHZULWH WKH IRUZDUGEDFNZDUG $3(6 HVWLPDWRU DV D0X0YLX!f4aOXLXffLSX-L8f Â£$3(VZ!Zf /?/D0O L0 FML Df 4 XT XLfD0O f$Â R!f ZKHUH 4FXXf 5 /?/ fÂ§ A + LL U!DLffWM! Z&Rf nZZfW! ZLO!f + f f ZLWK 5 GHQRWLQJ WKH IRUZDUGEDFNZDUG VDPSOH FRYDULDQFH PDWUL[ JLYHQ E\ f mrf Â f Â Â PAGE 52 DQG /HW /? fÂ§ = fÂ§ 9!ZLZf A\^KOfHaÂƒ8-OOO8Of K Â f = \f fff \^/? fÂ§ f fff \/ fÂ§ f fff \/L = f f DQG = -=rZKHUH GHQRWHV WKH H[FKDQJH PDWUL[ :H FDQ WKHQ UHZULWH f DQG f DV AZL:f =DÂOL/R!LXf f DQG AZLZf =DÂÂ ZLRf f %\ DSSO\LQJ WKH PDWUL[ LQYHUVLRQ OHPPD ZH REWDLQ 4fDfOAf 4.AfADf\f8fMXn8fADfnAf f /L/9! ZLZf4DLDfnFMDf ZKHUH 4DfAf 5 f // Â‘LS DL Df5B9! D!L FMf )RU QRWDWLRQDO FRQYHQLHQFH ZH VRPHWLPHV GURS WKH GHSHQGHQFH RQ FML DQG FM EHORZf +HQFH DQG D0[0Z L}ZLf4 FAL RfWMFAL DAf NL rPXP$ O:+4 4BA D0X0 /L/tPXP4 A /L/ WS+4[S /?/ LSQ4nnLS f D0L0:OLXf4 nD0L0MAOAf O0L0 4 nD0L0 0 4A + /L/LLS 4BL! f PAGE 53 ,W IROORZV WKDW f FDQ EH UHZULWWHQ DV }$3(6::f OPXP 4 >/L/L LIf 4 tPX04 OD0L0 D004 "Â‘ f 6LQFH 5 LV +HUPLWLDQ DQG SRVLWLYH GHILQLWH ZH FDQ REWDLQ DQ XSSHU WULDQJXODU PDWUL[ & E\ &KROHVN\ IDFWRUL]DWLRQ VXFK WKDW 5B &ULA&fA >@ /HW EUZLZf De ZLZD-& ? f GR!LZf fÂ§ 'DÂLÂZLZf f DQG HZLZf fÂ§ (DÂ_eMZL:f f ZKHUH &f+= f DQG ( &f+= f :H KDYH >@ D0L 0 4 D0L05a LAf E fÂ§ EA L-L FXfGLX FMf G E7F-L WXfHT!L FXfHQ D!fGD!L RMf /L/ fÂ§ __HFMLLXf__ f LS+ 4 WS 5 9! 77I; fÂ§ /?/ fÂ§ 9f 5A fÂ§ __GZLDf__ _GLDL DfHDL Zf /?/ fÂ§ __He-LXf__ B f PAGE 54 DQG 4 nDZ ?+ + 7` fÂ§ P 0L 0 D0L05 D0X0 f /[/ 9A5A LLE.AfIOE7-fHAnAfI A M //__HDDf__ f 1H[W ZH REVHUYH WKDW EZLXf GX!L FMf DQG HX!LXf FDQ EH FDOFXODWHG YLD ))7 6SHFLILFDOO\ ZH SDUWLWLRQ &B DQG ( DV IROORZV &B YHF^%L` YHF^%0O0` DQG ( a? 7 YHF^'L` YHF^'0O0` L7 f f f YHF^(` YHF^(0O0` ZKHUH %r &0O[0 f &/O;O DQG (r &LO;Â 7KHQ DJDLQ ZH GURS WKH GHSHQGHQFH RQ X[ DQG R!f D0 p D0 -YHFM%` DAU fYHF^%0O$Â` D0LAOD0 fff D0%00Dr0 f /LNHZLVH ZH KDYH DQG G 'DL_/D H (D r/O0 DA'LD O DÂ'0L0DO DI(LD D" (M f f /L(MLO fff D/LK00D/ 1RWH WKDW DA%ADA UHSUHVHQWV WKH GLVFUHWH )RXULHU WUDQVIRUP RI %IF DW D!LDf +HQFH $3(6 FDQ EH HIILFLHQWO\ LPSOHPHQWHG E\ ILUVW FDOFXODWLQJ EX!LXf GFMLR!f PAGE 55 DQG HXMXXMf YLD ))7 WKHQ XVLQJ (TXDWLRQV f f DQG f WR GHWHUn PLQH DAL 04W! nWS 4aOLS DQG DA!04atPXP UHVSHFWLYHO\ DQG ILQDOO\ XVLQJ f WR REWDLQ Â$3(6ALfAf 7KH VWUXFWXUH RI f LV VLPLODU WR WKDW RI f +RZHYHU WKH DPRXQW RI FRPSXWDWLRQ UHTXLUHG E\ WKH IRUPHU LV PXFK ODUJHU WKDQ WKDW E\ WKH ODWWHU 7KH UHDVRQ LV WKDW HYHQ WKRXJK "!X!LX!f DQG AXM ?Xf FDQ EH REWDLQHG E\ ))7 IRU HDFK FRLWXf SDLU ZH KDYH WR FRPSXWH WKH DGGLWLRQDO PDWUL[YHFWRU SURGXFWV 5W!R!LZf DQG 5Bn0OO0DLAf UHFDOO WKDW 5B H &0O0;0O0 bMf^XLXf DQG LSX!LXf f &0O0L;Of WR REWDLQ [Z Af DQG Xf 2Q WKH RWKHU KDQG E\ FRPSXWLQJ DQG ( ILUVW ZKLFK DUH FRPSXWHG RQO\ RQFHf ZH E\SDVV FDOFXODWLQJ VXFK PDWUL[YHFWRU SURGXFWV DQG VDYH D ODUJH DPRXQW RI FRPSXWDWLRQ 7KH ODUJHU WKH QXPEHU RI VDPSOHV LQ WKH IUHTXHQF\ GRPDLQ WKH PRUH WKH DPRXQW RI FRPSXWDWLRQ ZH ZLOO VDYH 7KHVH GLVFXVVLRQV DOVR DSSO\ WR WKH LPSOHPHQWDWLRQ RI &DSRQ ([WHQVLRQ WR &DSRQ 1RWH WKDW $3(6 EHFRPHV &DSRQ ZKHQ 4AAf LV UHSODFHG E\ 5 +HQFH WKH HIILFLHQW LPSOHPHQWDWLRQ RI &DSRQ FDQ UHDGLO\ EH DFKLHYHG E\ PRGLI\LQJ f DV IROORZV E7ZL:fGe-L:f m&DSRQ.ZDf /Oe __E:O!:f__ f 0RUH VSHFLILFDOO\ WKH HIILFLHQW LPSOHPHQWDWLRQ RI &DSRQ LV E\ XVLQJ f DQG f WR FDOFXODWH EDT R!f DQG GXT X!f UHVSHFWLYHO\ DQG WKHQ XVLQJ WKHP LQ f 6LQFH WKH DPRXQW RI FRPSXWDWLRQ UHTXLUHG WR FDOFXODWH ED!F'f LQ f GXLXff LQ f RU HDfLFXf LQ f LV DSSUR[LPDWHO\ WKH VDPH DQG FDOFXODWLQJ m$3(6 2$ !Af E\ f DQG G4DSQZL ORf E\ f DUH PXFK OHVV LQYROYHG WKDQ REWDLQLQJ EXLXf GXTXf DQG HXLXff WKH WRWDO DPRXQW RI FRPSXWDWLRQ UHTXLUHG E\ $3(6 LV DERXW WLPHV RI WKDW UHTXLUHG E\ &DSRQ DV YHULILHG E\ WKH QXPHULFDO DQG H[SHULPHQWDO H[DPSOHV LQ 6HFWLRQ PAGE 56 )RU VLPLODU UHDVRQV DV IRU $3(6 WKH LQWXLWLYH LPSOHPHQWDWLRQ RI &DSRQ JLYHQ LQ f LV FRPSXWDWLRQDOO\ PRUH LQYROYHG WKDQ WKH HIILFLHQW LPSOHPHQWDWLRQ RI &DSRQ SURSRVHG DERYH 1XPHULFDO DQG ([SHULPHQWDO ([DPSOHV :H SUHVHQW QXPHULFDO DQG H[SHULPHQWDO H[DPSOHV FRPSDULQJ WKH SHUIRUPDQFHV RI $3(6 DQG &DSRQ ZLWK WKH ))7 PHWKRGV >@ IRU 6$5 LPDJLQJ ,Q WKH IROORZLQJ H[DPSOHV ZH FKRRVH 0 1 DQG 0 Â“ IRU ERWK &DSRQ DQG $3(6 )RU WKH ZLQGRZHG ))7 PHWKRG ZH XVH WKH .DLVHU ZLQGRZ ZLWK SDUDPHWHU :H ILUVW FRQVLGHU 6$5 LPDJLQJ RI D VLPXODWHG 0,* DLUSODQH 7KH [ GDWD PDWUL[ ZDV SURYLGHG E\ WKH 1DYDO 5HVHDUFK /DERUDWRU\ 7KH [ 6$5 LPDJH REWDLQHG E\ XVLQJ ))7 ZLQGRZHG ))7 &DSRQ DQG $3(6 DUH VKRZQ LQ )LJXUHV Df WR Gf UHVSHFWLYHO\ :H QRWH WKDW &DSRQ DQG $3(6 RXWSHUIRUP WKH ))7 PHWKRGV 7KH QXPEHU RI IORSV UHTXLUHG E\ RXU HIILFLHQW ZD\V RI LPSOHPHQWLQJ &DSRQ DQG $3(6 DUH DERXW DQG WLPHV RI WKRVH UHTXLUHG E\ WKH ))7 PHWKRGV ZKLOH WKRVH UHTXLUHG E\ WKH LQWXLWLYH ZD\V RI LPSOHPHQWLQJ &DSRQ DQG $3(6 DUH DERXW DQG WLPHV UHVSHFWLYHO\ RI WKRVH UHTXLUHG E\ WKH ))7 PHWKRGV 7KDW LV WKH QXPEHU RI IORSV UHTXLUHG E\ WKH LQWXLWLYH ZD\V RI LPSOHPHQWLQJ &DSRQ DQG $3(6 DUH UHVSHFWLYHO\ DERXW DQG WLPHV RI WKRVH UHTXLUHG E\ RXU HIILFLHQW ZD\V RI LPSOHPHQWLQJ WKHP ,I ZH LQFUHDVH WKH VL]H RI WKH LPDJH WR [ DQG [ UHVSHFWLYHO\ WKH UDWLRV RI WKH QHHGHG IORSV EHWZHHQ WKH LQWXLWLYH ZD\V DQG RXU QHZ ZD\V IRU LPSOHPHQWDWLRQV DUH t IRU &DSRQ DQG t IRU $3(6 UHVSHFWLYHO\ :H QRZ FRQVLGHU DQ H[DPSOH RI 6$5 LPDJLQJ ZLWK H[SHULPHQWDO GDWD 7KH GDWD PDWUL[ LV [ DQG LV REWDLQHG IURP WKH H[SHULPHQWDO GDWD FROOHFWHG E\ RQH RI WKH WZR DSHUWXUHV RI WKH (5,0fV (QYLURQPHQWDO 5HVHDUFK ,QVWLWXWH RI 0LFKLJDQfVf '&6 ,)6$5 LQWHUIHURPHWULF 6$5f 7KH [ 6$5 LPDJH REWDLQHG E\ XVLQJ ' PAGE 57 ))7 ZLQGRZHG ))7 &DSRQ DQG $3(6 DUH VKRZQ LQ )LJXUHV Df WR Gf UHVSHFWLYHO\ $JDLQ &DSRQ DQG $3(6 RXWSHUIRUP WKH ))7 PHWKRGV 7KH QXPEHU RI IORSV UHTXLUHG E\ WKH LQWXLWLYH ZD\V RI LPSOHPHQWLQJ &DSRQ DQG $3(6 DUH UHVSHFWLYHO\ DERXW DQG WLPHV RI WKRVH UHTXLUHG E\ RXU HIILFLHQW ZD\V RI LPSOHPHQWLQJ WKHP ,I ZH LQFUHDVH WKH VL]H RI WKH LPDJH WR [ WKH UDWLRV RI WKH QHHGHG 0$7/$% IORSV EHWZHHQ WKH LQWXLWLYH ZD\V DQG RXU QHZ ZD\V RI LPSOHPHQWLQJ &DSRQ DQG $3(6 DUH DQG UHVSHFWLYHO\ 6XPPDU\ 7KLV FKDSWHU DGGUHVVHV WKH LPSOHPHQWDWLRQ RI WKH &DSRQ DQG $3(6 VSHFWUDO HVWLPDWRUV 7KH DPRXQW RI FRPSXWDWLRQ UHTXLUHG E\ $3(6 LV VKRZQ WR EH DERXW WLPHV WKDW UHTXLUHG E\ &DSRQ %\ XVLQJ D WHFKQLTXH SURSRVHG LQ WKLV FKDSWHU WKH FRPSXWDWLRQDO FRPSOH[LWLHV RI &DSRQ DQG $3(6 DUH VLJQLILFDQWO\ UHGXFHG PAGE 58 )LJXUH 6$5 LPDJHV REWDLQHG IURP WKH (5,0 GDWD E\ XVLQJ Df ZLQGRZHG ))7 Ff &DSRQ DQG Gf $3(6 ))7 Ef PAGE 59 &+$37(5 $03/,78'( (67,0$7,21 ,QWURGXFWLRQ &RQVLGHU WKH QRLVHFRUUXSWHG REVHUYDWLRQV RI FRPSOH[YDOXHG VLQXVRLGV >@ >@ [Qf nADNHM:NQ?YQf Q 1 f N ZKHUH DcW GHQRWHV WKH FRPSOH[ DPSOLWXGH RI WKH NWK VLQXVRLG KDYLQJ IUHTXHQF\ 8N 1 LV WKH QXPEHU RI DYDLODEOH GDWD VDPSOHV DQG YQf LV WKH REVHUYDWLRQ QRLVH ZKLFK LV FRPSOH[YDOXHG DQG DVVXPHG WR EH VWDWLRQDU\ DQG SRVVLEO\ FRORUHGf ZLWK ]HURPHDQ DQG ILQLWH XQNQRZQ 3RZHU 6SHFWUDO 'HQVLW\ 36'f :H DVVXPH WKDW ^Â!N`N L DUH NQRZQ ZLWK X!N A WRL IRU N A O 7KH SUREOHP RI LQWHUHVW LV WR HVWLPDWH ^DN`A IURP WKH REVHUYDWLRQV ^AQf`AA ,Q WKLV FKDSWHU ZH GHVFULEH D UHODWLYHO\ ODUJH QXPEHU RI PHWKRGV IRU VROYLQJ WKLV SUREOHP 6HFWLRQ GLVFXVVHV OHDVW VTXDUHV /6f PHWKRGV /6 PHWKRGV DUH ZLGHO\ XVHG IRU DPSOLWXGH HVWLPDWLRQ EHFDXVH WKH\ DUH VLPSOH DQG HDV\ WR LPSOHPHQW ,I ZH UHVWULFW RXUVHOYHV WR HVWLPDWLQJ RQO\ RQH DPSOLWXGH DW D WLPH WKHQ WKH /6 PHWKRG UHGXFHV WR WKH 'LVFUHWH )RXULHU 7UDQVIRUP ')7f RI WKH GDWD DW WKH IUHTXHQF\ RI WKH GHVLUHG VLQXVRLG DQG LV FRPSXWDWLRQDOO\ PRUH HIILFLHQW WKDQ WKH /6 PHWKRG WKDW HVWLPDWHV DPSOLWXGHV VLPXOWDQHRXVO\ 0RUHRYHU HVWLPDWLQJ RQH DPSOLWXGH DW D WLPH GRHV QRW QHFHVVDULO\ UHTXLUH H[DFW NQRZOHGJH RI WKH QXPEHU RI VLQXVRLGV LQ WKH GDWD DQG RI WKH IUHTXHQF\ ORFDWLRQ RI HDFK VLQXVRLG ZKLFK LV D GHVLUHG SURSHUW\ LQ VRPH DSSOLFDWLRQV 7KH GLVDGYDQWDJH KRZHYHU LV WKDW XVLQJ WKLV RQHDWDWLPH WHFKQLTXH LQ JHQHUDO JLYHV UDWKHU SRRU DPSOLWXGH HVWLPDWHV ZKHQ VRPH VLQXVRLGV WKDW PAGE 60 DUH RI LQWHUHVW WR XVf DUH FORVH WR RQH DQRWKHU 6WDWLVWLFDO DQDO\VHV WKDW FRPSDUH WKH PHULWV RI WKH WZR /6 PHWKRGV DUH DOVR SURYLGHG LQ 6HFWLRQ 6LQFH WKH /6 PHWKRGV FRPSOHWHO\ LJQRUH WKH FRUUHODWLRQ RI WKH REVHUYDWLRQ QRLVH WKH\ DUH LQ JHQHUDO VXERSWLPDO %\ VSOLWWLQJ WKH GDWD YHFWRU LQWR D QXPEHU RI VXEYHFWRUV WKH FRYDULDQFH PDWUL[ RI WKH QRLVHRQO\ SDUW RI WKH GDWD VXEYHFWRUV FDQ EH HVWLPDWHG ZKLFK PDNHV LW SRVVLEOH WR XVH D 0DUNRYOLNH HVWLPDWRU WKDW LV RSWLPDO LQ WKH FODVV RI :HLJKWHG /HDVW 6TXDUHV :/6f WHFKQLTXHV >@ :H GHVFULEH LQ 6HFWLRQ VHYHUDO ZD\V IRU HVWLPDWLQJ WKH DIRUHPHQWLRQHG FRYDULDQFH PDWUL[ ZKLFK OHDG WR GLIIHUHQW :/6 DPSOLWXGH HVWLPDWRUV $GGLWLRQDOO\ ZH VKRZ WKDW LI WKH UHVWULFWLRQ RI HVWLPDWLQJ RQH DPSOLWXGH DW D WLPH LV DJDLQ LPSRVHG ZH REWDLQ WZR :/6 DPSOLWXGH HVWLPDWRUV WKDW DUH HTXLYDOHQW WR WKH &DSRQ >@ >@ DQG $3(6 >@ PHWKRGV H[WHQVLYHO\ XVHG IRU VSHFWUDO DQDO\VLV 7KH REVHUYDWLRQ WKDW VRPH JHQHUDO VSHFWUDO HVWLPDWRUV VXFK DV &DSRQ DQG $3(6 FDQ EH XVHG WR VROYH WKH SUREOHP SRVHG LQ f PRWLYDWHG XV WR VHHN RWKHU UHODWLYHO\ VRSKLVWLFDWHG VSHFWUDO DQDO\VLV WHFKQLTXHV IRU DPSOLWXGH HVWLPDWLRQ %RWK &DSRQ DQG $3(6 EHORQJ WR WKH JHQHUDO FODVV RI LQWHUEDQN DSSURDFKHV WR VSHFWUDO HVWLPDWLRQ >@ ZKLFK LQYROYH VSOLWWLQJ WKH GDWD LQWR VXEYHFWRUV SDVVLQJ WKHP WKURXJK D VHW RI QDUURZEDQG ILOWHUV ILOWHUEDQNf ZKRVH FHQWHU IUHTXHQFLHV FRUUHVSRQG WR WKRVH WKDW DUH RI LQWHUHVW WR XV DQG ILQDOO\ HVWLPDWLQJ WKH VSHFWUDO GHQVLW\ IXQFWLRQ DW WKRVH IUHTXHQFLHV IURP WKH ILOWHUHG DQG KRSHIXOO\ VLJQDOHQKDQFHG GDWD $V RQH FDQ VHH WKH NH\ LVVXH RI ILOWHUEDQN DSSURDFKHV LV WKH GHVLJQ RI WKH ILOWHUV $ UHFHQW VWXG\ KDV VXJJHVWHG WKH FKRLFH RI PDWFKHG ILOWHUV ZKLFK JDYH ULVH WR WKH 0$WFKHG),OWHUEDQN 0$),f DSSURDFK WR VSHFWUDO HVWLPDWLRQ >@ (YHQ WKRXJK QHLWKHU &DSRQ QRU $3(6 ZDV GHULYHG LQ WKH 0$), IUDPHZRUN VHH >@ DQG >@ IRU WKHLU RULJLQDO GHULYDWLRQVf LW ZDV IRXQG WKDW ERWK DUH PHPEHUV RI WKH 0$), DSSURDFK >@ ,Q WKH OLJKW RI WKH ZRUN RI >@ ZH GHULYH LQ 6HFWLRQ D JHQHUDOL]HG 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ ,QWHUHVWLQJO\ HQRXJK ZH VKRZ WKDW XQGHU FHUWDLQ FLUFXPVWDQFHV 0$), PAGE 61 DPSOLWXGH HVWLPDWRUV KDYH HTXLYDOHQW IRUPV WR WKH :/6 PHWKRGV +RZHYHU WKH 0$), DSSURDFK LV PRUH JHQHUDO WKDQ WKH :/6 WHFKQLTXH LQ WKDW WKH ODWWHU LV D VSHFLDO FDVH RI WKH IRUPHU 7R VKRZ WKLV D QHZ 0$), DPSOLWXGH HVWLPDWRU WKDW GRHV QRW IDOO LQWR WKH :/6 FDWHJRU\ LV GHVFULEHG LQ 6HFWLRQ 2WKHU LQWHUHVWLQJ 0$), DPSOLWXGH HVWLPDWRUV PD\ H[LVW DQG DUH \HW WR EH GLVFRYHUHG $ FRPPRQ IHDWXUH RI WKH DPSOLWXGH HVWLPDWRUV FRQVLGHUHG LQ WKLV FKDSWHU LV WKDW QRQH RI WKHP PRGHOV WKH REVHUYDWLRQ QRLVH H[DFWO\ (YHQ VR DOO PHWKRGV DUH DV\PSWRWLFDOO\ VWDWLVWLFDOO\ HIILFLHQW WKDW LV WKH\ DOO DFKLHYH WKH &UDPU5DR %RXQG &5%f LQ ODUJH VDPSOHV +RZHYHU WKHLU ILQLWHVDPSOH SURSHUWLHV ZKLFK DUH RI SULPDU\ LQWHUHVW WR WKLV ZRUN DUH TXLWH GLIIHUHQW 6LQFH WKH ILQLWHVDPSOH DQDO\VLV LV LQWUDFWDEOH LQ PRVW FDVHV ZH XVH 0RQWH&DUOR VLPXODWLRQV LQ 6HFWLRQ WR FRPSDUH WKHVH PHWKRGV ZLWK RQH DQRWKHU 7KH DPSOLWXGH HVWLPDWLRQ SUREOHP LQ f RFFXUV LQ D YDULHW\ RI VLJQDO SURn FHVVLQJ DSSOLFDWLRQV VHH HJ >@ >@ DQG WKH UHIHUHQFHV WKHUHLQf ,Q 6HFWLRQ ZH GLVFXVV LWV DSSOLFDWLRQ WR V\VWHP LGHQWLILFDWLRQ :H VKRZ WKDW E\ XVLQJ DSSURSULn DWH DPSOLWXGH HVWLPDWRUV ZH FDQ DYRLG WKH LWHUDWLYH VHDUFK UHTXLUHG E\ WKH VWDQGDUG V\VWHP LGHQWLILFDWLRQ URXWLQHV VXFK DV WKH 2XWSXW (UURU 0HWKRG 2(0f >@ DQG DFKLHYH YHU\ JRRG SHUIRUPDQFH DW D XVXDOO\ UHGXFHG FRPSXWDWLRQDO ORDG ,Q FRQFOXGLQJ WKLV VHFWLRQ ZH LQWURGXFH WKH IROORZLQJ QRWDWLRQ WR GLVWLQJXLVK DPRQJ WKH YDULRXV DPSOLWXGH HVWLPDWRUV )RU LQVWDQFH /6(f GHQRWHV WKH /6 HVWLPDWRU WKDW GRHV QRW VSOLW WKH GDWD DQG KHQFH LW XVHV RQH GDWD fVQDSVKRWff XVHV QR SUHILOWHULQJ DQG HVWLPDWHV RQH DPSOLWXGH DW D WLPH /LNHZLVH 0$),/ .f GHQRWHV WKH 0$), HVWLPDWRU WKDW VSOLWV WKH GDWD LQWR / VXEYHFWRUV XWLOL]HV D EDQN RI SUHILOWHUV DQG HVWLPDWHV DPSOLWXGHV VLPXOWDQHRXVO\ 7KH UHPDLQLQJ DPSOLWXGH HVWLPDWRUV DUH VLPLODUO\ GHVLJQDWHG PAGE 62 /6 $PSOLWXGH (VWLPDWRUV :H FRQVLGHU WZR /6 PHWKRGV LQ WKLV VHFWLRQ QDPHO\ /6( .f DQG /6( f /6(IO .f 7KLV LV SHUKDSV WKH PRVW GLUHFW DSSURDFK /HW XV ZULWH WKH DYDLODEOH GDWD VHTXHQFH LQ WKH IROORZLQJ IRUP [f m Rn S rf fÂ§ HMZL HM8. m QOf WfÂ§+ HM1 fFO}L J M1OfXL. D. &6 K RU ZLWK REYLRXV GHILQLWLRQV [ ÂƒD Y f ZKLFK LV D OLQHDU UHJUHVVLRQ HTXDWLRQ 7KH /6 HVWLPDWH RI D LV Â£ ÂƒAÂƒAÂƒA[ f ZKHUH f+ GHQRWHV WKH FRQMXJDWH WUDQVSRVH 1RWH WKDW WKH QRLVH LV QRW PRGHOHG HYHQ WKRXJK LW PD\ EH FRUUHODWHG 'HVSLWH WKLV IDFW /6( .f LV DV\PSWRWLFDOO\ HIILFLHQW >@ $ UHODWLYHO\ VLPSOH PDQQHU WR VHH WKLV LV DV IROORZV )LUVW QRWH WKDW (^FW` fÂ§ D f ZKHUH (^` GHQRWHV WKH VWDWLVWLFDO H[SHFWDWLRQ 7KH 0HDQ 6TXDUHG (UURU 06(f RI Â•/ Â6 06(^T` FRYMR` ( ^R fÂ§ Df D fÂ§ RLf+` ÂƒÂƒfÂƒÂ:ÂƒÂƒÂÂÂƒf f ZKHUH Z f PAGE 63 1H[W VLQFH VHH HJ >@f -f6RAÂƒÂƒf ,n ZKHUH A GHQRWHV WKH 1 [ 1 LGHQWLW\ PDWUL[ DQG OLP Âƒ:Âƒf 9!RR 1 Â!^ZNf WKH DV\PSWRWLF 06( LV JLYHQ E\ f W!0 OLP,906(^D` 1fÂ§g c!^XNf f 8QGHU WKH PLOG DVVXPSWLRQ WKDW YQf LV FLUFXODUO\ V\PPHWULF *DXVVLDQ WKH &5% IRU D LV JLYHQ E\ VHH HJ >@f &5%^D` ÂƒL:BÂƒfB f 8VLQJ WKH IROORZLQJ UHVXOW VHH >@ RQFH DJDLQf OLP Âƒ+:BÂƒf 1RR 1 f /:f Mf WR Nf f ZH REWDLQ W!0 OLP 9&5%^W[` 1fÂ§!RR ZNf f ZKLFK FRLQFLGHV ZLWK f 5HPDUN ,W FDQ EH UHDGLO\ FKHFNHG IURP f DQG f WKDW LI YQf LV ZKLWH LH : a 1 WKHQ /6(5f LV VWDWLVWLFDOO\ HIILFLHQW IRU DOO 1 . PAGE 64 /6(ÂO2OO 6LQFH WKH REVHUYDWLRQ QRLVH YQf LV QRW PRGHOHG DQ LGHD WKDW UHGXFHV WKH FRPSXWDWLRQDO EXUGHQ TXLWH D ELW LV WR LQFOXGH fÂ§ VLQXVRLGV LQ WKH QRLVH WHUP DQG KHQFH HVWLPDWH RQO\ RQH DPSOLWXGH DW D WLPH ,Q VRPH VLJQDO SURFHVVLQJ DSSOLFDWLRQV WKH IUHTXHQFLHV ^XfN`N PD\ EH XQNQRZQ $ W\SLFDO ZD\ WR HVWLPDWH ERWK ^DN`b O DQG ^XN`e ZRXOG FRQVLVW RI HVWLPDWLQJ MXVW RQH DPSOLWXGH IRU YDU\LQJ IUHTXHQF\ X! DQG WKHQ GHWHFWLQJ WKH SHDNV LQ WKH VRREWDLQHG VSHFWUXP >@ >@ >@ $V VXFK WKH DVVXPSWLRQ PDGH LQ 6HFWLRQ WKDW ^/XN`N DUH NQRZQ D SULRUL PD\ EH UHOD[HG ZKHQ XVLQJ WKH RQHDWDWLPH WHFKQLTXH 7KHUH LV D VRPHZKDW VXEWOH SUREOHP ZLWK WKH DERYH WHFKQLTXH WKH VXP RI YQf DQG fÂ§ VLQXVRLGV QR ORQJHU KDV D ILQLWH 36' DQG KHQFH RQH RI WKH SUHYLRXVO\ PDGH DVVXPSWLRQV IDLOV 1HYHUWKHOHVV WKH LGHD VWLOO ZRUNV DV ORQJ DV QR WZR VLQXVRLGV WKDW DUH RI LQWHUHVWf DUH VSDFHG WRR FORVH WR RQH DQRWKHU DV VKRZQ EHORZ DQG ODWHU LQ 6HFWLRQ 7KH /6(f LV HDVLO\ GHULYHG DV Q ZKLFK LV UHFRJQL]HG DV WKH ')7 RI ^AQf`A DW XMN 7KH WZR HVWLPDWHV LQ f DQG f ZLOO EH FORVH WR RQH DQRWKHU LI ?FRN fÂ§ i! 1 9 NLN A Of >@ $Q DQDO\VLV RI /6(f UXQV DV IROORZV :LWKRXW ORVV RI JHQHUDOLW\ OHW XV FRQVLGHU f IRU N 7KH /6(f HVWLPDWH RI D? LV JLYHQ E\ DL Â£Â£f Â£A[ f ZKHUH 7 Â£ O HL:L B B B HM1LfZL f DQG ZKHUH f GHQRWHV WKH WUDQVSRVH 7DNLQJ WKH H[SHFWDWLRQ RI f \LHOGV f PAGE 65 ZKHUH D 8 &W. @7f DQG $ LV GHILQHG WKURXJK Â£ $ f +HQFH /6( f LV ELDVHG +RZHYHU LW LV DV\PSWRWLFDOO\ XQELDVHG WKDW LV LWV ELDV JRHV WR ]HUR DV 1 fÂ§} RRf :H QH[W FDOFXODWH WKH 06( RI RQ 06(^Â£L` Â£+Â£f D+ >$FWD $+ :f Â£Â£+Â£f f 0DNLQJ XVH RI f RQFH DJDLQ DORQJ ZLWK WKH IDFW WKDW $+D?IÂ“ fÂ§}f DV 1 fÂ§! RR ZH KDYH OLP $06(^GL` c!^O?f f 1RR +HQFH /6(f LV DOVR DV\PSWRWLFDOO\ HIILFLHQW 2Q WKH RWKHU KDQG LQ ILQLWH VDPSOHV f PD\ EH EHWWHU RU ZRUVH WKDQ f GHSHQGLQJ RQ WKH FKDUDFWHULVWLFV RI WKH VFHQDULR XQGHU VWXG\ 7KH IDFW WKDW f PD\ EH EHWWHU WKDQ f FRPHV DV QR VXUSULVH $V DQ H[DPSOH OHW XV DVVXPH WKDW WKH 6LJQDOWR1RLVH 5DWLR 615f LV KLJK 7KHQ WKH ELDV RI f GRPLQDWHV WKH YDULDQFH SDUW 2Q WKH RWKHU KDQG f KDV QR ELDV DQG LWV YDULDQFH ZLOO EH VPDOOHU WKDQ WKH ELDV RI f LI WKH 615 LV ODUJH HQRXJK &RQVHTXHQWO\ WKH 06( RI f ZLOO EH VPDOOHU WKDQ WKDW RI f 7KH IDFW WKDW f PD\ EH EHWWHU WKDQ f LV KRZHYHU D VXUSULVH )RU DQ H[DPSOH RI VXFK D FDVH DVVXPH 615 & DQG : 1 7KHQ IRU f 06(ORQ` m Â£AÂ£f f ZKHUHDV IRU f 06(^G` $A$f LL f ZKLFK FDQ EH PXFK ODUJHU WKDQ f HJ LI ?X!N fÂ§ :L_ a O1 IRU VRPH N f ,Q f >MLM GHQRWHV WKH LMWK HOHPHQW RI WKH PDWUL[ DUJXPHQW PAGE 66 1RWH WKDW IRU PRVW FDVHV RI LQWHUHVW /6($nf ZLOO JLYH PRUH DFFXUDWH DPSOLWXGH HVWLPDWHV WKDQ /6(f DQG WKDW WKH GLIIHUHQFH EHWZHHQ WKHVH WZR HVWLPDWRUV LV VPDOO IRU ODUJH 1 2Q WKH RWKHU KDQG /6(f LV FRPSXWDWLRQDOO\ PRUH HIILFLHQW WKDQ /6( .f VLQFH WKH PDWUL[ PXOWLSOLFDWLRQ DQG LQYHUVLRQ LQ f DUH DYRLGHG +HQFH /6(f PD\ VWLOO EH ZRUWK FRQVLGHULQJ :/6 $PSOLWXGH (VWLPDWRUV ,I ZH VSOLW WKH GDWD YHFWRU [ LQWR VXEYHFWRUV WKHQ WKH FRYDULDQFH PDWUL[ RI WKH QRLVH SDUW RI WKH VXEYHFWRUV PD\ EH HVWLPDWHG DQG FDQ KHQFH EH XVHG WR GHULYH DQ RSWLPDO :/6 HVWLPDWRU LH D 0DUNRYOLNH HVWLPDWRUf >@ ,Q WKLV VHFWLRQ ZH GHVFULEH D QXPEHU RI VXFK :/6 HVWLPDWRUV WKDW VSOLW WKH GDWD LQWR YHFWRUV RI VKRUWHU OHQJWK XWLOL]H QR SUHILOWHULQJ DQG HVWLPDWH HLWKHU RQH RU DPSOLWXGHV DW D WLPH :/6(I/ .f :H GHILQH WKH IROORZLQJ VXEYHFWRUV \ ZKHUH 7 [Of [ O f [^O 0 fÂ§ f / 1 0 / f f 7KH FKRLFH RI 0 0 FDQ EH FKRVHQ VPDOOHU WKDQ 6HH )LJXUH 0RUHRYHU ZKHQ 0 fÂ§ DOO :/6(/ .f UHGXFH WR /6( .ff RU HTXLYDOHQWO\ RI / LV GLVFXVVHG LQ 6HFWLRQ :H KDYH f L D[HA YOf \ HMZL H--. YO f HM0OfXQ HM^0O fXMN DSFHAr YO 0 f RU ZLWK REYLRXV QRWDWLRQ \ $V f f PAGE 67 $OWHUQDWLYHO\ ZH FDQ UHZULWH f DV \ $ L2O Hf ZKHUH $ $ SMZLL S-8.O $' :H ZLOO XVH f PRVWO\ IRU DQDO\VLV DQG f IRU HVWLPDWLRQ 7KH :/6 0DUNRYOLNHf HVWLPDWH RI D LQ f LV JLYHQ E\ D / L U / R / e$I4n\ ZKHUH 4 LV DQ HVWLPDWH RI 4 e^HHf` 7R HVWLPDWH 4 ZH PD\ SURFHHG DV IROORZV /HW 5 `(\\II: 2QH FDQ YHULI\ WKDW DV / fÂ§! RR 5 JRHV WR 5 $3$ 4 ZKHUH +HQFH RQH ZD\ WR HVWLPDWH 4 LV DV 4 5 $3$ f f f f f f f f ZKHUH 3 LV PDGH IURP VRPH LQLWLDO HVWLPDWHV RI ^DN`N ? REWDLQHG IRU LQVWDQFH YLD RQH RI WKH /6 DPSOLWXGH HVWLPDWRUV 7KH QHHG IRU LQLWLDO DPSOLWXGH HVWLPDWHV LV D PAGE 68 R GUDZEDFN RI 4 LQ f ,Q WKH IROORZLQJ ZH WU\ WR FLUFXPYHQW WKLV QHHG LQ WZR GLIIHUHQW ZD\V )LUVW ZH VKRZ D ZD\ WR VLPSOLI\ WKH :/6(/ .f WKDW XVHV f ZLWK f )URP f ZH KDYH WKDW 54B$ $3$A4A$ $ $,? f ZKHUH U 3$A4A$ ,r f )RU VXIILFLHQWO\ ODUJH 1 DQG 0 7 LV DSSUR[LPDWHO\ GLDJRQDO VLQFH $+4a$ LV VR VHH HJ ff &RQVHTXHQWO\ 4B$ 5B$U' m 5A'MU 5f[$U ,QVHUWLQJ f LQWR f \LHOGV REVHUYH WKDW 9+ FDQFHOV RXWf Q UU f H[ (LI5n$ / R /O 8 R f ZKLFK XQOLNH XVLQJ f ZLWK f GRHV QRW UHTXLUH DQ\ LQLWLDO HVWLPDWH RI ^H[r`A 7KH DPSOLWXGH HVWLPDWRU LQ f FDQ EH LQWHUSUHWHG DV DQ H[WHQVLRQ RI WKH &DSRQ DOJRULWKP LQ >@ >@ WR PXOWLSOH VLQXVRLGV $ GLIIHUHQW HVWLPDWH RI 4 FDQ EH REWDLQHG DV GHVFULEHG QH[W 2EVHUYH WKDW $3$K A>DND^XMNf`>DND^XNf@+ N N ZKHUH DFMf H f f :H FDQ XVH WKH YHFWRUV ^N`e O LQWURGXFHG DERYH WR UHZULWH f DV \ N f PAGE 69 )URP f ZH FDQ HVWLPDWH IN RQH DW D WLPH YLD /6 DV /B IW (\AHB:LAJAf N f / 1RWH WKDW ZH FRXOG HVWLPDWH DOO ^N`N L VLPXOWDQHRXVO\ YLD /6 ZKLFK KRZHYHU DSSHDUV WR SHUIRUP HYHQ ZRUVH WKDQ XVLQJ f HVSHFLDOO\ IRU VPDOO 1f 7KH XVH RI f LQ f DQG f OHDGV WR WKH IROORZLQJ HVWLPDWH RI 4 4 5 f N 7KH :/6(/ .f WKDW XVHV f ZLWK f GRHV QRW UHTXLUH DQ\ LQLWLDO HVWLPDWH RI ^DN`e Y ,W LV DQ H[WHQVLRQ RI WKH $3(6 DOJRULWKP LQ >@ WR PXOWLSOH VLQXVRLGV ZLWK NQRZQ IUHTXHQFLHV 5HPDUN :H QRWH WKDW HNf DQG H=f LQ f DUH FRUUHODWHG IRU N A Of ZKLFK LPSOLHV WKDW f LV VXERSWLPDO DV LW WDNHV LQWR DFFRXQW RQO\ WKH FRUUHODWLRQ EHWZHHQ WKH HOHPHQWV RI H=f EXW LJQRUHV WKH FRUUHODWLRQ EHWZHHQ HOf DQG HIFf IRU N A Of PAGE 70 f DUH UHFRJQL]HG WR KDYH WKH VDPH IRUP DV WKH &DSRQ >@ >@ DQG UHVSHFWLYHO\ WKH $3(6 >@ VSHFWUDO HVWLPDWRUV 7KH WZR HVWLPDWRUV ZHUH GHULYHG LQ >@ >@ E\ D GLIIHUHQW DSSURDFK QDPHO\ WKH 0$), DSSURDFK ZKLFK ZH ZLOO FRQVLGHU LQ D JHQHUDOL]HG IRUP LQ WKH QH[W VHFWLRQ ,W LV LQWHUHVWLQJ WKDW WKH DERYH WZR DPSOLWXGH HVWLPDWRUV ZKLOH ERWK DV\PSn WRWLFDOO\ HIILFLHQW DQG KHQFH HTXLYDOHQWf KDYH TXLWH GLIIHUHQW ILQLWHVDPSOH SURSHUWLHV 6SHFLILFDOO\ LW ZDV VKRZQ LQ >@ >@ WKDW f LV ELDVHG GRZQZDUG ZKHUHDV f LV XQELDVHG ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQf DQG LQ JHQHUDO KDV D EHWWHU SHUn IRUPDQFH WKDQ WKH IRUPHU 0$), $PSOLWXGH (VWLPDWRUV ,Q WKLV VHFWLRQ ZH GHULYH D JHQHUDOL]HG 0$), DSSURDFK WR DPSOLWXGH HVWLPDn WLRQ /HW &.[0 EH D PDWUL[ HDFK URZ RI ZKLFK LV D )LQLWH ,PSXOVH 5HVSRQVH ),5f ILOWHU IRU VRPH 0 \HW WR EH VSHFLILHGf 7KH 0$), LGHD FDQ EH H[SODLQHG DV IROORZV Df 'HVLJQ VR WKDW ZKHQ DSSOLHG WR ^\=f` LW PD[LPL]HV WKH 615 DW WKH ILOWHU RXWSXWV Ef (VWLPDWH WKH DPSOLWXGHV IURP WKH ILOWHUHG GDWD ZKRVH 615 VKRXOG EH KLJKHU WKDQ WKDW LQ WKH UDZ GDWDf E\ HJ WKH /6 RU :/6 WHFKQLTXH 0DWKHPDWLFDOO\ + FDQ EH REWDLQHG DV IROORZV + DUJPD[WU +K4+fB+$3$f+ + f f*HQHUDOL]HG 615f ZKHUH + LV FRQVWUDLQHG LQ D ZD\ WKDW LV VSHFLILHG ODWHU LQ SDUWLFXODU WR JXDUDQWHH WKDW + LV ILQLWHf DQG WUf GHQRWHV WKH WUDFH RI D PDWUL[ /HW ; +4+f+nn4 f PAGE 71 ZKHUH f GHQRWH WKH +HUPLWLDQ VTXDUH URRW RI WKH SRVLWLYH GHILQLWH PDWUL[ DUJXn PHQW 2EVHUYH WKDW ; LV VHPLXQLWDU\ LH ;+; O5 f 7KH FRVW IXQFWLRQ LQ f FDQ QRZ EH UHZULWWHQ DV WU [TDSDKT[@ f ,W IROORZV IURP WKH 3RLQFDU VHSDUDWLRQ WKHRUHP RU WKH JHQHUDOL]HG 5D\OHLJK TXRWLHQW WKHRUHPf >@ WKDW 5 PD[ ;N &7$3$A4f f N ZKHUH ^$IFf`MA GHQRWH WKH HLJHQYDOXHV RI WKH PDWUL[ EHWZHHQ WKH SDUHQWKHVHV RUn GHUHG VXFK WKDW $L $ ! IXUWKHUPRUH WKH FROXPQV RI WKH PD[LPL]LQJ ; DUH HTXDO WR WKH HLJHQYHFWRUV FRUUHVSRQGLQJ WR ^$IF`A 1H[W QRWH WKDW SRVWPXOWLSO\LQJ ; E\ DQ\ XQLWDU\ PDWUL[ RI DSSURSULDWH GLPHQVLRQV \LHOGV DQRWKHU YDOLG VROXWLRQ IRU ; 2QH VXFK VROXWLRQ KDYLQJ D VLPSOH IRUP FDQ EH REWDLQHG DV IROORZV 2EVHUYH WKDW UDQN Tf$3$K4f f ZKLFK LPSOLHV WKDW ZH FDQQRW LPSURYH WKH JHQHUDOL]HG 615 E\ FKRRVLQJ VLQFH ;5 2Q WKH RWKHU KDQG WKH ODUJHU WKH WKH PRUH ILOWHUHG GDWD ZLOO EH DYDLODEOH IRU DPSOLWXGH HVWLPDWLRQ +HQFH ZH FKRRVH . ,Q VXFK D FDVH WKH PD[LPL]LQJ ; LV JLYHQ E\ f ; 4$7 f PAGE 72 ZKHUH 7 GHQRWHV VRPH QRQVLQJXODU PDWUL[ WKDW PDNHV ; VHPLXQLWDU\ 2QH VXFK 7 ,6 7 $+ &7 $fa f +HQFH ; 4$$K4$f f :H QH[W REVHUYH WKDW + 4; f VDWLVILHV f &RQVHTXHQWO\ ZH KDYH + 4a$$K4a$fa f 7KH ILQDO VWHS LV WR REVHUYH WKDW SRVWPXOWLSO\LQJ + E\ D QRQVLQJXODU PDWUL[ GRHV QRW FKDQJH WKH JHQHUDOL]HG 615 FULWHULRQ 7KHQ LW IROORZV LPPHGLDWHO\ WKDW + 4A$Â$A4A$f f PD[LPL]HV WKH JHQHUDOL]HG 615 DQG LW DOVR VDWLVILHV WKH FRQVWUDLQW +A$ ,r f 7KH FRQVWUDLQW f VD\V WKDW HDFK URZf ILOWHU LQ ++ SDVVHV RQH VLQXVRLG XQGLVn WRUWHG DQG FRPSOHWHO\ DQQLKLODWHV WKH RWKHUV )URP f DQG f WKH ILOWHUHG GDWD FRUUHVSRQGLQJ WR f LV JLYHQ E\ ] ++\f 'rD ++H=f 'ÂD L=f / f 7KH FRYDULDQFH PDWUL[ RI Lf FDQ EH HVWLPDWHG DV +4+ $K4$fB f ,W IROORZV WKDW WKH :/6 0DUNRYOLNHf HVWLPDWH RI D LQ f LV JLYHQ E\ D /fÂ§ 'L$LI4f $f' / U /fÂ§ ;@'I$4$f$4$f$L4\ ;$I4n$ / A$I4n\ Lf f PAGE 73 &O ZKLFK VKRZV WKDW 0$),/ .f :/6(/ .f f 7KH 0$), LQWHUSUHWDWLRQ RI WKH :/6 PHWKRG DIIRUGHG E\ WKH DERYH DQDO\VLV LV LQWHUHVWLQJ ,Q SDUWLFXODU LW PDNHV D FOHDU FRQQHFWLRQ EHWZHHQ XVLQJ WKH 0$), DQG WKH :/6 WHFKQLTXHV IRU DPSOLWXGH HVWLPDWLRQ 7KH 0$), DSSURDFK LV KRZHYHU PRUH JHQHUDO WKDQ WKH :/6 WHFKQLTXH $V DQ H[DPSOH ZH GHULYH D QHZ 0$), DPSOLWXGH HVWLPDWRU WKDW GRHV QRW EHORQJ WR WKH :/6 FODVV DV IROORZV /HW ]N^Of DQG YN^Of GHQRWH WKH IFWK HOHPHQW RI ]9f DQG UHVSHFWLYHO\ XOf LQ f 7KHQ ]N^Of DNHA XNOf N f 7KH DERYH HTXDWLRQV DUH UHODWHG WR RQH DQRWKHU RQO\ YLD WKH FRUUHODWLRQ EHWZHHQ YN^Of DQG XSOf IRU N A Sf ,I ZH LJQRUH WKH FRUUHODWLRQ WKHQ WKH 0$),/ f HVWLPDWH RI WKH DN YLD /6 LV JLYHQ E\ /aO tN a-A]NOfHA f A 8QOLNH WKH &DSRQ f DQG $3(6 f HVWLPDWRUV ZKLFK FDQ DOVR EH VKRZQ WR EH PHPEHUV RI WKH 0$),/L) f FODVV >@f WKH DERYH 0$),/ f HVWLPDWRU GRHV UHTXLUH WKH NQRZOHGJH RI WKH QXPEHU DQG IUHTXHQFLHV RI WKH VLQXVRLGV ZKLFK PDNHV LW EHKDYH PRUH OLNH D 0$),/ .f HVWLPDWRU ,Q SDUWLFXODU LW SHUIRUPV TXLWH ZHOO IRU FDVHV ZKHUH VRPH VLQXVRLGV DUH FORVHO\ VSDFHG DV ZLOO EH VHHQ LQ 6HFWLRQ 2WKHU LQWHUHVWLQJ 0$), DPSOLWXGH HVWLPDWRUV PD\ EH GHYLVHG E\ XVLQJ VRPH RWKHU FKRLFHV RI + LQ OLHX RI WKH RQH JLYHQ LQ f DV WKH VROXWLRQ WR f LV QRW XQLTXH 6SHFLILFDOO\ RQH PD\ LQWURGXFH FHUWDLQ XQLWDU\ WUDQVIRUP RQ WKH + LQ f RU FKRRVH . RU UHSODFH WKH ,N LQ f E\ DQRWKHU QRQVLQJXODU PDWUL[ ZKLFK DOO OHDG WR VROXWLRQV WKDW DUH GLIIHUHQW IURP f )XUWKHUPRUH RQH FRXOG HYHQ FKDQJH WKH FULWHULRQ LQ f WR DQRWKHU UHDVRQDEOH GHILQLWLRQ RI WKH fJHQHUDOL]HG 615f +RZHYHU VXFK YDULDWLRQV RQ WKH WKHPH RI 0$), DUH EH\RQG WKH VFRSH RI WKH SUHVHQW ZRUN PAGE 74 1XPHULFDO ([DPSOHV ,Q ZKDW IROORZV ZH LQYHVWLJDWH WKH SHUIRUPDQFHV RI WKH YDULRXV DPSOLWXGH HVWLPDWRUV GHVFULEHG LQ WKH SUHYLRXV VHFWLRQV )RU QRWDWLRQDO VLPSOLFLW\ ZH ZLOO UHIHU WR WKHVH PHWKRGV DV IROORZV f /6( /6(f XVLQJ f f /6(. /6(.f XVLQJ f f &DSRQO :/6(/f XVLQJ f f $3(6 :/6(/ f XVLQJ f f &DSRQ. :/6(/)f XVLQJ f f $3(6. :/6(/ .f XVLQJ f DORQJ ZLWK f f 0$), 0$),/ f XVLQJ f DORQJ ZLWK f :H ZLOO FRPSDUH WKHVH PHWKRGV ZLWK RQH DQRWKHU DV ZHOO DV WKH &5% JLYHQ LQ f 6LQFH DOO WKHVH PHWKRGV DUH DV\PSWRWLFDOO\ HIILFLHQW ZH RQO\ FRQVLGHU WKH FDVH ZKHQ 1 LV UHODWLYHO\ VPDOO 6SHFLILFDOO\ ZH FKRRVH 1 7KH GDWD FRQVLVW RI WKUHH FRPSOH[ VLQXVRLGV FRUUXSWHG E\ D FRPSOH[ *DXVVLDQ QRLVH YQf WR EH VSHFLILHGf [Qf VQf YQf Q fÂ§ 1 fÂ§ f ZKHUH VQf n`7DNHMrK f IF L 7KH IUHTXHQFLHV RI WKH VLQXVRLGV DUH L DQG $OVR DL Hn RÂ H-LU DQG H-U $OO H[DPSOHV DUH EDVHG RQ 0RQWH&DUOR VLPXODWLRQV 7KH 06( ILJXUHV VKRZQ LQ ZKDW IROORZV DUH REWDLQHG DV 06(^FEJRR a DN X Â f PAGE 75 )LJXUH 36' RI WKH WHVW GDWD WKDW FRQVLVW RI WKUHH VLQXVRLGV DQG DQ $5f QRLVH SURFHVV ZKHUH Â£IF]f LV WKH HVWLPDWH RI DN GHULYHG LQ WKH ]WK VLPXODWLRQ UXQ (VWLPDWLRQ 3HUIRUPDQFH YHUVXV 615 )LUVW ZH FRQVLGHU WKH FDVH ZKHUH YQf LV FRORUHG 0RUH H[DFWO\ YQf LV GHn VFULEHG E\ WKH IROORZLQJ $XWR5HJUHVVLYH $5f SURFHVV YQf XQ fÂ§ f HQf f ZLWK HQf EHLQJ D FRPSOH[ ZKLWH *DXVVLDQ QRLVH ZLWK ]HURPHDQ DQG YDULDQFH D 7KH 36' RI WKH WHVW GDWD LV VKRZQ LQ )LJXUH ZKHUH D fÂ§ 7KH ORFDO 615 RI WKH NWK VLQXVRLG LV GHILQHG DV >@ ,U\ 1?FÂX 3 615IF ORJ m ORJLR ÂUSU f +IfGI Q-N! 1RWH WKH RFFXUUHQFH RI 1 LQ WKH DERYH 615 IRUPXOD )RU WKRVH PHWKRGV WKDW GHSHQG RQ 0 ZH FKRRVH 0 1 JLYLQJ / VHH 6HFWLRQ IRU D VWXG\ RI WKH HIIHFW RI 0 RQ WKH SHUIRUPDQFHf )LJXUH Df VKRZV WKH 06(V RI WKH VHYHQ DPSOLWXGH HVWLPDWRUV IRU DORQJ ZLWK WKH FRUUHVSRQGLQJ &5% DV WKH 615 YDULHV $V RQH FDQ VHH $3(6 $3(6. DQG 0$), DUH YHU\ FORVH WR WKH &5% ZKLOH /6(. ZKLFK LJQRUHV WKH QRLVH FRUn UHODWLRQ LV HYLGHQWO\ DZD\ IURP WKH &5% &DSRQ. DOVR GHYLDWHV IURP WKH &5% IRU PAGE 76 PRVW 615V 7KH UHDVRQ LV WKDW WKH DSSUR[LPDWLRQ PDGH LQ f LV YDOLG RQO\ IRU ODUJH 1 DQG 0 ZKLFK LV QRW WKH FDVH LQ WKLV H[DPSOH )LJXUH Df DOVR VKRZV WKDW ERWK /6( DQG &DSRQO DUH LQFRQVLVWHQW LQ 615f 7KHLU LQFRQVLVWHQF\ LV QRW VXUn SULVLQJ EHFDXVH ERWK DUH ELDVHG HVWLPDWRUV 5HFDOO WKDW WKH ELDV RI /6( DV JLYHQ LQ f GRHV QRW YDQLVK XQOHVV 1 JRHV WR LQILQLW\ 6LPLODUO\ &DSRQO LV DOZD\V ELDVHG GRZQZDUGf IRU ILQLWH 1 >@ >@ )LJXUH Ef VKRZV WKH FRXQWHUSDUW FXUYHV IRU DT 7KH UHVXOWV IRU D DUH RPLWWHG EHFDXVH WKH\ UHVHPEOH WKRVH IRU RTf 1RWH WKDW fÂ§ L ZKLFK LV VPDOOHU WKDQ O1 m WKH )RXULHU UHVROXWLRQ OLPLW 7KH SHUIRUPDQFH GHJUDGHV IRU DOO HVWLPDWRUV XQGHU VWXG\ HVSHFLDOO\ IRU /6( &DSRQO DQG $3(6 ZKLFK HVWLPDWH RQO\ RQH DPSOLWXGH DW D WLPH $V VKRZQ LQ )LJXUH Ef /6( DQG &DSRQO HVVHQWLDOO\ IDLO IRU DOO 615V FRQVLGHUHG GXH WR WKHLU ODUJH 06(V $3(6 LV QR ORQJHU FORVH WR WKH &5% EXW XQOLNH WKH SUHYLRXV WZR HVWLPDWRUV LW VWLOO DSSHDUV WR EH FRQVLVWHQW LQ 615f $V LQ )LJXUH Df &DSRQ. DJDLQ GHYLDWHV DZD\ IURP WKH &5% DW KLJK 615V ,W DSSHDUV WKDW WKH DSSUR[LPDWLRQ PDGH LQ f LQWURGXFHV D ELDV DW VPDOO 1 DQG 0f WKDW PD\ EH QHJOLJLEOH DW ORZ 615V EXW GRPLQDWHV WKH YDULDQFH DW KLJK 615V 7KH ELDV GRHV QRW GLVDSSHDU DV WKH 615 LQFUHDVHV ZKLFK FDXVHV WKH GLYHUJHQFH RI &DSRQ. IURP WKH &5% $3(6. SHUIRUPV TXLWH ZHOO IRU KLJK 615V KRZHYHU LW LV QRW YHU\ VWDEOH DW ORZ 615V GXH WR ODUJH YDULDQFHf 7KH EHVW HVWLPDWRU LQ WKLV H[DPSOH LV 0$), 7KH NQRZOHGJH RI WKH QXPEHU DQG ORFDWLRQV RI WKH VLQXVRLGV ZKLFK WKH RWKHU RQHDWDWLPH HVWLPDWRUV PD\ VSDUH EXW LV LQGLVSHQVDEOH WR 0$), DSSHDUV WR SOD\ DQ LPSRUWDQW UROH LQ LWV JRRG SHUIRUPDQFH LQ WKH FXUUHQW FDVH WKDW IDLOV WKH RWKHU RQHDWDWLPH HVWLPDWRUV $V VWDWHG LQ 6HFWLRQ /6(. LV VWDWLVWLFDOO\ HIILFLHQW LH LW DFKLHYHV WKH &5% IRU DQ\ 1 ZKHQ WKH REVHUYDWLRQ QRLVH LV ZKLWH 7R VHH KRZ WKH RWKHU VXERSWLPDO LQ ILQLWH VDPSOHVf PHWKRGV SHUIRUP LQ VXFK D FDVH ZH FRQVLGHU DQ H[DPSOH ZKLFK LV VLPLODU WR WKH SUHYLRXV RQH H[FHSW WKDW YQf LV UHSODFHG E\ D ]HURPHDQ FRPSOH[ ZKLWH PAGE 77 Df Ef )LJXUH (PSLULFDO 06(V DQG WKH &5% YHUVXV ORFDO 615 ZKHQ 1 0 DQG WKH REVHUYDWLRQ QRLVH LV FRORUHG DQ $5f SURFHVVf Df Ef m PAGE 78 *DXVVLDQ QRLVH 7KH 615 LV GHILQHG LQ WKH VDPH PDQQHU DV LQ f )LJXUHV Df DQG Ef VKRZ WKH 06(V RI WKH DPSOLWXGH HVWLPDWHV RI DQG UHVSHFWLYHO\ DT DQG WKH FRUUHVSRQGLQJ &5% DV WKH 615 LQFUHDVHV $V RQH FDQ VHH WKH $3(6 $3(6. DQG 0$), HVWLPDWHV RI D DUH DJDLQ YHU\ FORVH WR WKH &5% ZKHUHDV IRU DQ DOO VXERSWLPDO PHWKRGV VXIIHU IURP VRPH SHUIRUPDQFH ORVV DV FRPSDUHG WR WKH RSWLPDO /6(. DQG \HW WKH GLIIHUHQFHV EHWZHHQ /6(. DQG 0$), IRU DOO 615V FRQVLGHUHG KHUH DUH IDLUO\ VPDOO $ EULHI VXPPDU\ EDVHG RQ WKH SUHYLRXV VWXG\ LV DV IROORZV $3(6 LV UHFRPn PHQGHG LQ DSSOLFDWLRQV ZKHUH LW LV NQRZQ D SULRUL WKDW QR WZR VLQXVRLGV DUH FORVHO\ VSDFHG VHH HJ WKH DSSOLFDWLRQ GLVFXVVHG LQ WKH QH[W VHFWLRQf RU ZKHQ WKH FORVHO\ VSDFHG VLQXVRLGV DUH RI QR LQWHUHVW 7KH UHDVRQ WR SUHIHU $3(6O WR $3(6. RU 0$), LQ VXFK FDVHV LV WKDW WKH IRUPHU LV PRUH IOH[LEOH WKDQ WKH ODWWHU WZR VLQFH $3(6O GRHV QRW QHFHVVDULO\ UHTXLUH WKH NQRZOHGJH RI WKH VLQXVRLGDO IUHTXHQFLHV ,Q WHUPV RI FRPn SXWDWLRQDO FRVW $3(6O DQG 0$), DUH VLPLODU WR RQH DQRWKHU DQG ERWK DUH VLPSOHU WKDQ $3(6. :KHQ LW LV GHVLUHG WR HVWLPDWH FORVHO\ VSDFHG VLQXVRLGV LQ FRORUHG QRLVH KRZHYHU 0$), PD\ EH SUHIHUUHG ,Q JHQHUDO ZH GR QRW UHFRPPHQG WKH XVH RI &DSRQO VLQFH LW KDV D FRPSXWDWLRQDO FRPSOH[LW\ VLPLODU WR WKDW RI $3(6O EXW LV ELDVHG $OWKRXJK ZH GLG QRWLFH WKDW &DSRQ. JLYHV FORVHWR&5% SHUIRUPDQFH DW YHU\ ORZ 615V LQ PRVW FDVHV RI LQWHUHVW RWKHU PHWKRGV OLNH $3(6O RU 0$), PD\ EH SUHIHUUHG /6(. LV VWDWLVWLFDOO\ HIILFLHQW DQG PD\ EH SUHIHUUHG ZKHQ WKH REVHUYDWLRQ QRLVH LV ZKLWH LQ FDVHV ZKHUH WKH ZKLWH QRLVH DVVXPSWLRQ LV LQYDOLG LW LV SUHIHUDEOH WR XVH $3(6O RU 0$), /6( JLYHV FRPSDUDWLYHO\ UDWKHU SRRU HVWLPDWLRQ DFFXUDF\ EXW LV FRPSXWDWLRQDOO\ TXLWH VLPSOH 7KH SHUIRUPDQFH GLIIHUHQFHV VWDWHG VR IDU RFFXU RQO\ ZKHQ 1 LV UHODWLYHO\ VPDOO $V 1 LQFUHDVHV DOO PHWKRGV WHQG WR WKH &5% LQn GHSHQGHQW RI WKH QRLVH FRUUHODWLRQ +HQFH ZKHQ 1 LV VXIILFLHQWO\ ODUJH /6( VKRXOG EH SUHIHUUHG EHFDXVH RI LWV FRPSXWDWLRQDO VLPSOLFLW\ PAGE 79 Ef )LJXUH (PSLULFDO 06(V DQG WKH &5% YHUVXV ORFDO 615 ZKHQ 1 0 DQG WKH REVHUYDWLRQ QRLVH LV ZKLWH Df Ef D? PAGE 80 7KH (IIHFW RI 0 $OO :/6 DQG 0$), DPSOLWXGH HVWLPDWRUV VWXGLHG LQ WKLV FKDSWHU GHSHQG RQ WKH FKRLFH RI 0 WKH VXEYHFWRU OHQJWK ,W LV NQRZQ WKDW DV 0 LQFUHDVHV DOO RI WKHP FDQ EHWWHU GHDO ZLWK WKH FDVH RI FORVHO\ VSDFHG VLQXVRLGV EXW WKHLU VWDWLVWLFDO VWDELOLW\ LQ JHQHUDO GHFUHDVHV >@ +HQFH WKHUH LV D WUDGHRII WR WR NHSW LQ PLQG ZKHQ FKRRVLQJ 0 1RWH WKDW 0 VKRXOG DOVR EH VPDOOHU WKDQ 1 RWKHUZLVH WKH HVWLPDWHG FRYDULDQFH PDWUL[ ZLOO EH UDQN GHILFLHQW 7KH IROORZLQJ H[DPSOH H[DPLQHV WKH HIIHFW RI 0 RQ WKH SHUIRUPDQFHV RI WKHVH HVWLPDWRUV /6( DQG /6(. GR QRW GHSHQG RQ 0 DQG DUH WKXV QRW FRQVLGHUHG LQ WKLV H[DPSOH 7KH VFHQDULR LV VLPLODU WR WKH ILUVW H[DPSOH $5 QRLVHf H[FHSW WKDW ZH IL[ D a ZKLFK FRUUHVSRQGV WR D ORFDO 615 RI G% IRU WKH ILUVW VLQXVRLG DW IL f DQG G% IRU WKH WKLUG VLQXVRLG DW f 0 LV YDULHG IURP WR IRU DOO HVWLPDWRUV H[FHSW 0$), ZKLFK UHTXLUHV WKDW 0 VHH f DQG ff 7KH 06(V RI WKH DPSOLWXGH HVWLPDWHV RI D DQG UHVSHFWLYHO\ RS DQG WKH FRUUHVSRQGLQJ &5%V DUH VKRZQ LQ )LJXUHV Df DQG Ef $V FDQ EH VHHQ IURP WKHVH ILJXUHV DOO HVWLPDWRUV DUH VHQVLWLYH WR WKH FKRLFH RI 0 WR D VPDOOHU RU ODUJHU H[WHQW :KHQ QR VLQXVRLGV DUH FORVH WR WKH RQH EHLQJ HVWLPDWHG VXFK DV WKH WKLUG VLQXVRLG LQ WKLV H[DPSOH $3(6O $3(6. DQG 0$), SHUIRUP TXLWH ZHOO IRU D ZLGH UDQJH RI 0 )RU WKH PRUH GLIILFXOW FDVH DV VKRZQ LQ )LJXUH Ef WKH FKRLFH RI 0 EHFRPHV YHU\ FULWLFDO %DVHG RQ RXU HPSLULFDO H[SHULHQFH D UXOH RI WKXPE IRU WKH FKRLFH RI 0 LV JLYHQ LQ 7DEOH $SSOLFDWLRQ WR 6\VWHP ,GHQWLILFDWLRQ &RQVLGHU WKH OLQHDU GLVFUHWHWLPH V\VWHP GHVFULEHG E\ WKH IROORZLQJ HTXDWLRQ >@ [ULf +] OfXQfYQf Q 1 fÂ§ f PAGE 81 R en /-aR &' OLRn FU &' F & 1 R 2I &RORUHG 1RLVH 7 f D &DSRQ ; f f f Â‘ ; $3(6 B f &DSRQ. 2 f f R $3(6. L fff f 0$), &5% , , / 6XEYHFWRU /HQJWK 0 Df 1 D I &RORUHG 1RLVH L L L LW UXQ Â’ D &DSRQO ; f; $3(6 &DSRQ. M R $3(6. 2 f 0$), B &5% n fÂ§ U r f [ Â‘' f[ fÂ§ R Rn H R fnn U L L L / L L 6XEYHFWRU /HQJWK 0 Ef )LJXUH (PSLULFDO 06(V DQG WKH &5% YHUVXV 0 ZKHQ 1 DQG WKH REVHUYDWLRQ QRLVH LV FRORUHG DQ $5f SURFHVV ZLWK D f Df D Ef D? PAGE 82 (VWLPDWRU 0 $3(6 1$ 0 1 $3(6. 1$ 0 1 0$), 1 0 9 &DSRQO RU &DSRQ. 1 0 1$ 7DEOH &KRLFH RI 0 IRU WKH :/6 DQG 0$), DPSOLWXGH HVWLPDWRUV ZKHUH WKH LQSXW XQf LV D VLQXVRLGDO SURELQJf VLJQDO XQf AONH@8NQ Q N O DQG WKH WUDQVIHU IXQFWLRQ LV UDWLRQDO +X %] nf EL] O ET] O $^]aOf DL] DS]a3 :H DVVXPH WKDW S T f f f (YHQ LI S DQG T ZHUH XQNQRZQ FRXOG VWLOO EH FKRVHQ VXIILFLHQWO\ ODUJH WR VDWLVI\ f 7KH SUREOHP RI LQWHUHVW LQ WKLV VHFWLRQ LV WR HVWLPDWH ^DÂ`A DQG ^A`\ IURP 6\VWHP ,GHQWLILFDWLRQ 8VLQJ $PSOLWXGH (VWLPDWLRQ 7KH FRPPRQO\XVHG 2XWSXW (UURU 0HWKRG 2(0f GRHV QRW PRGHO YQf DQG REWDLQV HVWLPDWHV RI ^DM`Â DQG ^EM`M O E\ PLQLPL]LQJ WKH FULWHULRQ 1fÂ§O A2(0D Ef +Qf a +^]aOfX^Qf f fÂ§ ZKHUH D > IOO DS @7 DQG E > E[ ET @7Â‘ /HW DrDEf MN+H-8-Nf f PAGE 83 )RU VXIILFLHQWO\ ODUJH 1 VR WKDW WKH WUDQVLHQW UHVSRQVH LQ WKH RXWSXW FDQ EH QHJOHFWHGf WKH FRVW IXQFWLRQ &R(0DfEf LV DSSUR[LPDWHO\ HTXLYDOHQW WR 1O &G DEf e Q [Qf DEfH-, IF L f 7KH PHWKRG WKDW ZH SURSRVH IRU HVWLPDWLQJ D DQG E LV EDVHG RQ f DQG FRQVLVWV RI WZR VWHSV )LUVW HVWLPDWH ^DN`N] LQ DQ XQVWUXFWXUHGQRQSDUDPHWULF IRUP f 7KHQ ILW ^mÂD Ef`A WR WKH DPSOLWXGH HVWLPDWHV REWDLQHG LQ WKH SUHYLRXV VWHS E\ WDNLQJ LQWR DFFRXQW WKH VWDWLVWLFDO YDULDQFH RI WKH ODWWHU ,Q ZKDW IROORZV ZH GHWDLO WKH DERYH WZR VWHSV 6WHS 8VH DQ DSSURSULDWH DPSOLWXGH HVWLPDWRU WR REWDLQ HVWLPDWHV ^Â£N`N RI ^RLN`N L IURP WKH PHDVXUHPHQWV ^AQf`A $3(6O PD\ EH UHFRPPHQGHG LQ WKLV FDVH EHFDXVH ZH KDYH FRQWURO RYHU WKH SURELQJ VLJQDO DQG ZH KDYH QR UHDVRQ WR FKRRVH DQ\ RI WKH VLQXVRLGV WRR FORVH WR RQH DQRWKHU 7KH ODUJHVDPSOH YDULDQFH RI WKH HVWLPDWHG DPSOLWXGHV ^tN`b O LV SURSRUWLRQDO WR ^If>R-Nf`N L VHH 6HFWLRQ f 7R REWDLQ HVWLPDWHV RI ^c!^ANf`N Y ZH FDQ ILUVW FDOFXODWH YQf fÂ§ [Qf fÂ§ \@ Â£NH/NQ Q 1 fÂ§ f N DQG WKHQ XWLOL]H HLWKHU D SDUDPHWULF RU D QRQSDUDPHWULF 36' HVWLPDWRU >@ >@ RQ f WR REWDLQ ^!XNf`N 6SHFLILFDOO\ LQ WKH H[DPSOHV JLYHQ LQ 6HFWLRQ ZH XVH WKH &DSRQ 36' HVWLPDWRU >@ >@ >@ DOVR VHH 6HFWLRQ f ZKLFK GHWHUPLQHV ^.AIFf`L L DV !^XNf 0 N O. f ZKHUH DXNf LV GHILQHG LQ f DQG 5YY LV WKH VDPSOH FRYDULDQFH PDWUL[ RI WKH HVWLPDWHG QRLVH YHFWRUV Y YOf YO f YO 0 f / f PAGE 84 WKDW LV LB 5YY A YfY+f f A 6WHS 2EWDLQ HVWLPDWHV RI ^DÂ` E\ PLQLPL]LQJ &DEf AAfÂ§ ?DN DIFDEf_ f r L QXNf 7R GR VR ZH FDQ XVH D KRVW RI PHWKRGV SURYLGHG WKDW ZH KDYH JRRG LQLWLDO HVWLPDWHV RI D DQG E 7R REWDLQ VXFK HVWLPDWHV DQG WKHQ PLQLPL]H f ZH DVVXPH WKDW S DQG T DUH NQRZQ 6WDQGDUG WHFKQLTXHV IRU V\VWHP RUGHU GHWHUPLQDWLRQ FDQ EH IRXQG LQ HJ >@ >@f :H SLFN XS WKH S T ODUJHVW ^Gr` LI WKH 615 LV ORZ DQ DOWHUQDWLYH LV WR FKRRVH WKRVH ^Dr` WKDW KDYH WKH ODUJHVW UDWLR _GMW_AZIHf DVVXPLQJ WKDW M!XfNf ZDV HVWLPDWHGf DQG GHILQH D FULWHULRQ PDGH IURP WKH FRUUHVSRQGLQJ WHUPV RI f 34 ADEf AAfÂ§ ?DN DIFDEf_ f MIF L .YNf ZKHUH ZH KDYH DVVXPHG IRU QRWDWLRQDO VLPSOLFLW\ WKDW ^Gr`A DUH WKH S T FKRVHQ DPSOLWXGHV 1RZ WKH PLQLPL]DWLRQ RI f LV VLPSOH ,QGHHG DOPRVW DOZD\V RQH FDQ FKRRVH D DQG E WR VDWLVI\ Gr DIFDEf $ S T f (TXDWLRQ f LV HTXLYDOHQW WR fÂ§7HAf 5HAIFf N S T f N ZKLFK FDQ EH UHZULWWHQ DV D OLQHDU V\VWHP RI S T HTXDWLRQV ZLWK S T XQNQRZQV ^FLLEM` 7KDW V\VWHP ZLOO JHQHUDOO\ KDYH D XQLTXH VROXWLRQ WKDW PDNHV f HTXDO WR ]HUR DQG ZKLFK WKHUHIRUH JLYHV RXU LQLWLDO HVWLPDWHV RI ^DÂM` $V VKRZQ LQ WKH IROORZLQJ QXPHULFDO H[DPSOHV WKH LQLWLDO HVWLPDWHV DUH XVXn DOO\ TXLWH JRRG +HQFH RQH FDQ HYHQ VNLS WKH VWHS RI PLQLPL]LQJ f WR VDYH FRPSXWDWLRQV PAGE 85 5HPDUN $FFRUGLQJ WR WKH ([WHQGHG ,QYDULDQFH 3ULQFLSOH (;,3f >@ WKH HVWLPDWHV RI ^DLEM` REWDLQHG E\ PLQLPL]LQJ f DFKLHYH WKH &5% DV\PSWRWLFDOO\ DQG KHQFH WKH\ KDYH D EHWWHU DV\PSWRWLF DFFXUDF\ WKDQ WKH 2(0 HVWLPDWHV ZKHQHYHU YQf LV FRORUHG ,W DOVR IROORZV IURP WKLV REVHUYDWLRQ WKDW LQ WKH FDVH RI $7 S T WKH HVWLPDWHV REWDLQHG IURP f DUH DV\PSWRWLFDOO\ HIILFLHQW 7KLV ODWWHU UHVXOW RI D VRPHZKDW OLPLWHG LQWHUHVW GXH WR WKH UHTXLUHPHQW WKDW fÂ§ S Tf ZDV ILUVW SURYHG LQ >@ LQ D PXFK PRUH FRPSOLFDWHG ZD\ 1XPHULFDO ([DPSOHV 7KH IROORZLQJ H[DPSOHV DVVXPH WKDW S DQG T DUH NQRZQ WR IDFLOLWDWH SHUIRUn PDQFH FRPSDULVRQ ,W LV UHDVRQDEOH WR GR VR VLQFH ERWK 2(0 DQG WKH SURSRVHG PHWKRG XVH VLPLODU WHFKQLTXHV WR GHWHUPLQH WKH PRGHO RUGHUV $OVR ZH DGRSW WKH VWUDWHJ\ WR FKRRVH WKH S T ODUJHVW ^RUr` LQ 6WHS RI WKH SURSRVHG PHWKRG ([DPSOH 7KH V\VWHP FRQVLGHUHG LQ WKLV H[DPSOH LV JLYHQ E\ f ZLWK $A]f = ] f DQG %]af ] ]f f 7KH SURELQJ VLJQDO LV JLYHQ E\ XQf FRVUQf FRVUQf FRVUQf FRVUQf FRVUQf Q 1 fÂ§ f :H FRQVLGHU XVLQJ D UHDOYDOXHG SURELQJ VLJQDO EHFDXVH WKLV LV WKH XVXDO FDVH LQ SUDFWLFH $ VXEWOH TXHVWLRQ DULVHV DV WKH DPSOLWXGH HVWLPDWLRQ WHFKQLTXHV GLVFXVVHG LQ WKH SUHYLRXV VHFWLRQV DOO DVVXPH WKDW WKH VLQXVRLGV DUH FRPSOH[YDOXHG 2QH PLJKW LPSRVH FHUWDLQ FRQMXJDWH V\PPHWU\ FRQVWUDLQW DQG GHULYH VLPLODU WHFKQLTXHV WKDW DUH VSHFLILFDOO\ WDLORUHG IRU UHDOYDOXHG VLQXVRLGDO DPSOLWXGH HVWLPDWLRQ VR WKDW LI PAGE 86 O? fÂ§ fÂ§&2 WKH HVWLPDWRUV ZLOO JLYH RQ PAGE 87 RU 0$), EXW DW D VLJQLILFDQWO\ LQFUHDVHG FRPSXWDWLRQDO FRVW 'XH WR WKLV REVHUYDn WLRQ ZH GR QRW UHFRPPHQG XVLQJ WKLV DSSURDFK LH PLQLPL]LQJ f IRU UHILQHG HVWLPDWLRQ DFFXUDF\ 2WKHU PRUH VRSKLVWLFDWHG WHFKQLTXHV IRU V\VWHP LGHQWLILFDWLRQ VHH HJ >@ >@f PD\ EH SUHIHUUHG LQ WKDW HYHQW )LJXUH Ff DOVR VKRZV WKDW DV FRPSDUHG WR 2(0 WKHUH LV OLWWOH FRPSXWDWLRQDO DGYDQWDJH RI XVLQJ WKH LQLWLDO HVWLPDWHV REWDLQHG E\ $3(6 DQG 0$), 7KH UHDVRQ PD\ EH WKDW WKH V\VWHP LQ WKLV H[DPSOH LV TXLWH VLPSOH LW KDV ZKLWH RXWSXW HUURUV HWFf DQG DSSDUHQWO\ 2(0 UHDFKHV FRQYHUJHQFH LQ D UHODWLYHO\ VPDOO QXPEHU RI LWHUDWLRQV )RU D PRUH FRPSOH[ V\VWHP VXFK DV WKH RQH XVHG LQ WKH QH[W H[DPSOH 2(0 PD\ QHHG PRUH LWHUDWLRQV WR FRQYHUJH ,W VKRXOG EH PHQWLRQHG WKDW ZH GLG QRW SURJUDP RXU PHWKRG YHU\ FDUHn IXOO\ DQG KHQFH RXU FRGH LV XQOLNHO\ WR EH DV HIILFLHQW DV WKH 2(0 FRGH LQ 0$7/$% 5HJDUGLQJ WKH HVWLPDWLRQ DFFXUDF\ ZH VKDOO VWUHVV WKDW LQ WKH FXUUHQW FDVH ZKHUH WKH QRLVH Y^Qf LV ZKLWH 2(0 FRLQFLGHV ZLWK WKH RSWLPDO 0D[LPXP /LNHOLKRRG 0HWKRG 0/0f >@ >@ :KHQ YQf LV FRORUHG 2(0 LV QR ORQJHU 0/0 ,Q WKDW FDVH WKH LQLWLDO V\VWHP SDUDPHWHU HVWLPDWHV REWDLQHG E\ $3(6 RU 0$), PD\ RXWSHUIRUP WKRVH E\ 2(0 DV LQ IDFW VKRZQ LQ WKH QH[W H[DPSOH 5HFDOO WKDW /6(. LV VWDWLVWLFDOO\ HIILFLHQW ZKHQ WKH REVHUYDWLRQ QRLVH LV ZKLWH 7KHQ RQH PLJKW ZRQGHU ZK\ WKH LQLWLDO HVWLPDWHV JLYHQ E\ /6(. PD\ EH QRWDEO\ ZRUVH LQ VXFK D FDVH WKDQ WKRVH JLYHQ E\ $3(6 RU 0$), DV KDSSHQHG LQ WKH SUHYLRXV H[DPSOH HVSHFLDOO\ ZKHQ 1 LV VPDOOf 7KH UHDVRQ LV WKDW WKH WUDQVLHQW UHVSRQVH RI WKLV V\VWHP FDQQRW EH QHJOHFWHG IRU VPDOO 1 7R VKRZ WKLV WKH 36' RI [Qf LV HVWLPDWHG E\ XVLQJ WKH &DSRQ 36' HVWLPDWRU ZLWK 1 DQG 0 DQG LV SORWWHG LQ )LJXUH ,W VKRZV WZR H[WUD SHDNV ZKLFK EHKDYH OLNH WZR VLQXVRLGVf DW s 7KH H[WUD SHDNV DUH DWWULEXWHG WR WKH UHVSRQVH RI WKH V\VWHP ZKLFK KDV SROHV DW H'Unf WR WKH LQLWLDO FRQGLWLRQV 6LQFH LW LV HVVHQWLDO IRU /6(. WR KDYH WKH DFFXUDWH NQRZOHGJH RI WKH QXPEHU DQG IUHTXHQFLHV RI WKH VLQXVRLGV IUHTXHQFLHV WR JLYH UHOLDEOH DPSOLWXGH HVWLPDWHV LWV SHUIRUPDQFH LQ WKH SUHYLRXV H[DPSOH LV FRQVLGHUDEO\ PAGE 88 Df &RPSXWDWLRQDO &RPSOH[LW\ 'DWD /HQJWK 1 Ff Ef )LJXUH $YHUDJHG 506(V DQG WKH QXPEHU RI IORSV YHUVXV 1 IRU WKH ILUVW V\VWHP ZKHQ WKH REVHUYDWLRQ QRLVH LV ZKLWH D f DQG 0 IRU $3(6O DQG 0$), Df 506( RI DSDUDPHWHUV Ef 506( RI SDUDPHWHUV Ff 1XPEHU RI IORSV PAGE 89 )LJXUH 36' HVWLPDWH RI WKH RXWSXW RI WKH ILUVW V\VWHP FRUUXSWHG E\ ZKLWH QRLVH ZLWK R DQG 1 GHWHULRUDWHG 2Q WKH RWKHU KDQG WKH DERYH NQRZOHGJH LV QRW QHFHVVDULO\ QHHGHG E\ $3(6 DQG KHQFH LWV SHUIRUPDQFH LV QRW DIIHFWHG 8QOLNH $3(6 0$), GRHV UHTXLUH WKLV NQRZOHGJH PAGE 90 D3DUDPHWHUV E3DUDPHWHUV &RPSXWDWLRQDO &RPSOH[LW\ Y f9 f ZLWK /6(. m m f ZLWK $3(6 fR f ZLWK 0$), R f ZLWK $3(6 2(0 4 Â‘D 3(0 'DWD /HQJWK 1 Ff )LJXUH $YHUDJHG 506(V DQG WKH QXPEHU RI IORSV YHUVXV 1 IRU WKH VHFRQG V\VWHP ZKHQ WKH REVHUYDWLRQ QRLVH LV FRORUHG DQ $5f SURFHVV ZLWK D f DQG 0 IRU $3(6 DQG 0$), Df 506( RI DSDUDPHWHUV Ef 506( RI SDUDPHWHUV Ff 1XPEHU RI IORSV RI IORSV DV 1 LQFUHDVHV $V RQH FDQ VHH WKH LQLWLDO V\VWHP SDUDPHWHU HVWLPDWHV JLYHQ E\ $3(6 RU 0$), DUH VLJQLILFDQWO\ EHWWHU WKDQ WKRVH JLYHQ E\ 2(0 DQG \HW WKH IRUPHU WZR DUH FRPSXWDWLRQDOO\ PRUH HIILFLHQW WKDW WKH ODWWHU 6XPPDU\ 7KLV FKDSWHU H[DPLQHV WKH SUREOHP RI DPSOLWXGH HVWLPDWLRQ RI VLQXVRLGDO VLJn QDOV LQ FRORUHG QRLVH 7KUHH JHQHUDO FODVVHV RI HVWLPDWRUV QDPHO\ WKH /6 :/6 DQG 0$), DSSURDFKHV WR DPSOLWXGH HVWLPDWLRQ DUH GLVFXVVHG ,W LV VKRZQ WKDW XQGHU PAGE 91 FHUWDLQ FLUFXPVWDQFHV WKH 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ LV HTXLYDOHQW WR WKH :/6 DSSURDFK DQG \HW WKH IRUPHU LV PRUH JHQHUDO DQG LQFOXGHV WKH ODWWHU DV D VSHFLDO FDVH 7KH DPSOLWXGH HVWLPDWRUV XQGHU GLVFXVVLRQ FDQ EH IXUWKHU FDWHJRUL]HG GHSHQGLQJ RQ ZKHWKHU WKH\ HVWLPDWH RQH DPSOLWXGH DW D WLPH RU DOO DPSOLWXGHV VLPXOn WDQHRXVO\ 0$), RU :/6 PHWKRGV VXFK DV $3(6 $3(6. DQG 0$), LQ JHQHUDO JLYH PRUH DFFXUDWH DPSOLWXGH HVWLPDWHV IRU VLQXVRLGV LQ FRORUHG QRLVH 0HWKRGV WKDW HVWLPDWH RQO\ RQH DPSOLWXGH DW D WLPH VXFK DV $3(6O PRVWO\ GR QRW UHTXLUH WKH H[DFW NQRZOHGJH RI WKH QXPEHU DQG ORFDWLRQV RI WKH VLQXVRLGV DQG KHQFH DUH PRUH UREXVW WKDQ WKRVH WKDW HVWLPDWH DOO DPSOLWXGHV VLPXOWDQHRXVO\ $V DQ DSSOLFDWLRQ H[DPSOH D V\VWHP LGHQWLILFDWLRQ DSSOLFDWLRQ XVLQJ VLQXVRLGDO SURELQJ VLJQDOV LV GLVFXVVHG $ QHZ WHFKQLTXH IRU V\VWHP LGHQWLILFDWLRQ LV SUHVHQWHG WKDW FDQ DYRLG LWHUDWLYH VHDUFKHV WKURXJK ILWWLQJ WKH V\VWHP SDUDPHWHUV WR WKH RXWSXW DPSOLWXGH HVWLPDWHV ,W LV VKRZQ WKDW E\ XVLQJ WKLV WHFKQLTXH ZLWK DSSURSULDWH DPSOLWXGH HVWLPDWRUV VXFK DV $3(6O RU 0$), ZH FDQ REWDLQ UHVXOWV WKDW DUH JHQHUDOO\ EHWWHU WKDQ WKRVH FRUUHVSRQGLQJ WR WKH ZLGHO\XVHG LWHUDWLYH 2(0 \HW XVXDOO\ DW D UHGXFHG FRPSXWDWLRQDO FRVW PAGE 92 &+$37(5 &$321 (67,0$7,21 2) &29$5,$1&( 6(48(1&(6 ,QWURGXFWLRQ &RYDULDQFH VHTXHQFH HVWLPDWLRQ LV D XELTXLWRXV WDVN LQ GLJLWDO VLJQDO SURFHVVn LQJ $ VWDQGDUG WHFKQLTXH IRU HVWLPDWLQJ WKH FRYDULDQFH VHTXHQFHV LV WKH VRFDOOHG WKH VWDQGDUG VDPSOH FRYDULDQFH HVWLPDWRU 7KH VWDQGDUG FRYDULDQFH HVWLPDWHV DUH FRQVLVWHQW SURYLGHG WKDW WKH JLYHQ VLJQDOV DUH HUJRGLF WR WKH VHFRQG RUGHU +RZHYHU WKHUH LV D PDMRU FRQFHUQ RI XVLQJ WKH VWDQGDUG HVWLPDWRU GXH WR WKH XQUHDOLVWLF ZLQn GRZLQJ RI WKH REVHUYHG GDWD LW DVVXPHV WKDW LV LW DVVXPHV WKDW WKH GDWD EH\RQG WKH REVHUYHG GXUDWLRQ HLWKHU LV ]HUR RU UHSHDWV LWVHOI SHULRGLFDOO\ 3DUWO\ IRU WKLV UHDVRQ WKHUH KDYH EHHQ VHYHUDO DWWHPSWV LQ WKH OLWHUDWXUH WR GHULYH PRUH DFFXUDWH FRYDULDQFH HVWLPDWHV WKDQ WKH VWDQGDUG RQHV $ QRWDEOH H[DPSOH LV WKH DSSURDFK EDVHG RQ WKH %XUJ DXWRUHJUHVVLYH $5f VSHFWUDO HVWLPDWRU +RZHYHU WKH VRREWDLQHG FRYDULDQFH HVWLPDWRU ZDV IRXQG HYHQ OHVV DFFXUDWH WKDQ WKH VWDQGDUG VDPSOH FRYDULDQFH HVWLPDn WRU >@ 7R EH PRUH H[DFW WKH %XUJ DSSURDFK ZDV VKRZQ WR KDYH ODUJHU YDULDQFHV WKDQ WKH VWDQGDUG PHWKRG $QRWKHU DSSURDFK ZKLFK KDV JHQHUDWHG D ZKROH QHZ UHVHDUFK GLUHFWLRQ UHOLHV RQ WKH PD[LPXP OLNHOLKRRG 0/f SULQFLSOH >@ +RZHYHU WKH 0/ HVWLPDWLRQ RI FRYDULDQFH VHTXHQFHV LV D FRPSXWDWLRQDOO\ LQYROYHG SUREOHP ZKLFK GRHV QRW KDYH D FORVHGIRUP VROXWLRQ 7KH VROXWLRQ JLYHQ LQ >@ LV LWHUDWLYH DQG QRW JXDUDQWHHG WR EH JOREDOO\ RSWLPXP $SSDUHQWO\ WKHUH H[LVW QR FRPSHOOLQJ DOWHUQDWLYHV WR WKH VWDQGDUG PHWKRG WKDW FDQ EH UHFRPPHQGHG IRU JHQHUDO XVH ,Q WKLV FKDSWHU ZH SUHVHQW D QHZ PHWKRG QDPHO\ WKH &DSRQ PHWKRG >@ IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ 7KH &DSRQ PHWKRG REWDLQV WKH FRYDULDQFH VHTXHQFH HVWLPDWHV E\ )RXULHU LQYHUWLQJ WKH &DSRQ SRZHU VSHFWUDO GHQVLW\ 36'f HVWLPDWHV PAGE 93 7KHUH DUH EDVLFDOO\ WZR &DSRQ 36' HVWLPDWRUV UHIHUUHG WR DV &DSRQ >@ >@ DQG &DSRQ >@ KHUHLQ :H ILQG WKDW ZKLOH &DSRQ LV FDSDEOH RI ILQHU VSHFWUDO UHVROXWLRQ DURXQG WKH SHDNV RI D VSHFWUXP LW LV JHQHUDOO\ D JOREDOO\ SRRUHU VSHFWUDO HVWLPDWRU WKDQ &DSRQ :H KHQFH FRQFHQWUDWH RXU LQWHUHVW RQ &DSRQ IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ LQ WKLV FKDSWHU 6LQFH WKH &DSRQ VSHFWUD LH WKH 36' HVWLPDWHV DUH VKRZQ WR EH HTXLYDOHQW WR $5 RU DXWRUHJUHVVLYH PRYLQJDYHUDJH $50$f VSHFWUD WKH LQYHUVLRQ SURFHGXUH IRU FRPSXWLQJ WKH H[DFW FRYDULDQFH VHTXHQFHV FRUUHVSRQGLQJ WR WKH &DSRQ VSHFWUD FDQ EH LPSOHPHQWHG LQ D UDWKHU FRQYHQLHQW ZD\ 1RWH WKDW WKH FDOFXODWLRQ RI WKH FRYDULDQFH VHTXHQFHV FRUUHVSRQGLQJ WR WKH &DSRQ VSHFWUD LV DQ LQWHUHVWLQJ SUREOHP E\ LWVHOIf :H DOVR SUHVHQW DQ ))7EDVHG DSSUR[LPDWH PHWKRG WR FRPSXWH WKH FRYDULDQFH VHTXHQFHV IURP WKH &DSRQ VSHFWUD ,W KDV EHHQ IRXQG WKDW WKH DSSUR[LPDWH PHWKRG SURYLGHV FRYDULDQFH HVWLPDWHV WKDW DUH DOPRVW LGHQWLFDO ZLWK WKRVH REWDLQHG E\ WKH H[DFW PHWKRG ZKLOH WKH FRPSXWDWLRQDO FRPSOH[LW\ LV JUHDWO\ UHGXFHG 2XU SULPDU\ LQWHUHVW LV WR DSSO\ WKH &DSRQ PHWKRG DV ZHOO DV WKH VWDQGDUG DSSURDFK WR $50$ VLJQDOV 7R WKDW HQG D IHZ $50$ VLJQDOV ZLWK W\SLFDO SROH DQG ]HUR ORFDWLRQV DUH VWXGLHG LQ RXU QXPHULFDO H[DPSOHV 7KH VWXGLHV VKRZ WKDW FRQVLGHUDEOH LPSURYHPHQWV DUH DWWDLQHG E\ WKH QHZ &DSRQ PHWKRG 7KH &DSRQ FRYDULDQFH HVWLPDWLRQ PHWKRG FDQ EH UHDGLO\ XVHG LQ PDQ\ DSSOLn FDWLRQV 2QH LPSRUWDQW FODVV LV WKH $50$ VSHFWUDO HVWLPDWLRQ 6LQFH PRVW $50$ VSHFWUDO HVWLPDWRUV UHO\ RQ WKH PAGE 94 EHWWHU $5 FRHIILFLHQW HVWLPDWHV DUH REWDLQHG E\ XVLQJ WKH &DSRQ FRYDULDQFH HVWLn PDWHV WKDQ E\ WKH VWDQGDUG RQHV $QRWKHU DSSOLFDWLRQ GLVFXVVHG LQ WKLV FKDSWHU LV WKH PRYLQJDYHUDJH 0$f PRGHO RUGHU GHWHUPLQDWLRQ E\ PDNLQJ XVH RI WKH &DSRQ DQG WKH VDPSOH FRYDULDQFH HVWLPDWHV ZKHUH ZH ILQG WKDW EHWWHU SHUIRUPDQFH LV DFKLHYHG E\ WKH IRUPHU ,W VKRXOG EH PHQWLRQHG WKDW ZH FDQ REWDLQ $3(6 FRYDULDQFH VHTXHQFH HVWLn PDWHV VLPLODUO\ E\ )RXULHU LQYHUWLQJ WKH $3(6 36' HVWLPDWHV +RZHYHU LQ VSLWH RI WKH HYLGHQW GLIIHUHQFH RI WKH &DSRQ DQG $3(6 FRPSOH[ VSHFWUDO HVWLPDWHV IRU GLVFUHWH VSHFWUDO FRPSRQHQWV WKHLU SHUIRUPDQFHV LQ FRQWLQXRXV 36' HVWLPDWLRQ DUH VLPLODU WR RQH DQRWKHU ZKLFK LPSOLHV WKDW WKH $3(6 FRYDULDQFH HVWLPDWHV DUH VLPLODU WR WKH &DSRQ FRYDULDQFH HVWLPDWHV $V VXFK WKH $3(6 FRYDULDQFH HVWLPDWLRQ LV QRW GLVFXVVHG KHUHLQ 6WDQGDUG &RYDULDQFH (VWLPDWRU DQG 2XWORRN :LWK QR RWKHU DVVXPSWLRQV PDGH RQ WKH VLJQDO XQGHU VWXG\ H[FHSW IRU DVVXPn LQJ WKH VHFRQG RUGHU HUJRGLFLW\ WKHUH DUH WZR ZD\V WR REWDLQ WKH VWDQGDUG VDPSOH FRYDULDQFHV RI WKH VLJQDO QDPHO\ WKH ELDVHG DQG WKH XQELDVHG FRYDULDQFH HVWLPDWRUV +RZHYHU WKH ELDVHG FRYDULDQFH HVWLPDWRU LV PRUH FRPPRQO\ XVHG VLQFH LW SURYLGHV VPDOOHU PHDQVTXDUHG HUURUV 06(f WKDQ WKH XQELDVHG RQH DQG JXDUDQWHHV WKH FR YDULDQFH HVWLPDWHV WR EH SRVLWLYH VHPLGHILQLWH >@ 7KH ELDVHG VDPSOH FRYDULDQFH HVWLPDWRU RI D ZLGHVHQVH VWDWLRQDU\ VLJQDO ZLWK ]HURPHDQ KDV WKH IRUP -9IF f N ZKHUH ^\Qf`b DUH WKH REVHUYHG GDWD VDPSOHV 1 LV WKH QXPEHU RI VDPSOHV INf GHQRWHV WKH HVWLPDWH RI WKH FRYDULDQFH IXQFWLRQ UNf LV WKH ODUJHVW ODJ GHVLUHG 1 fÂ§ f DQG Â‘fr GHQRWHV WKH FRPSOH[ FRQMXJDWH 1RWH WKDW f LV DV\PSWRWLFDOO\ XQELDVHG PAGE 95 7KH HVWLPDWRU JLYHQ LQ f LV FRQVLVWHQW LI WKH HUJRGLFLW\ DVVXPSWLRQ LV VDWLVn ILHG $ VWXG\ RQ ZKHWKHU WKH VWDQGDUG VDPSOH FRYDULDQFH HVWLPDWRU LV DOVR DV\PSWRWn LFDOO\ VWDWLVWLFDOO\ HIILFLHQW LH ZKHWKHU LW DV\PSWRWLFDOO\ DFKLHYHV WKH &UDPU5DR %RXQG &5%f KDV EHHQ XQGHUWDNHQ LQ >@ /HW \Wf EH DQ $50$STf VLJQDO ,I S T WKHQ WKH VDPSOH FRYDULDQFH HVWLPDWH UNf LV DV\PSWRWLFDOO\ VWDWLVWLFDOO\ HIILn FLHQW LI DQG RQO\ LI N S fÂ§ T? LQ SDUWLFXODU IRU $5 SURFHVVHV RI RUGHU S UNf LV DV\PSWRWLFDOO\ HIILFLHQW IRU N S EXW LQHIILFLHQW IRU DOO RWKHU N ,I S T QRQH RI UNf LV DV\PSWRWLFDOO\ HIILFLHQW LQ SDUWLFXODU QRQH RI UNf LV DV\PSWRWLFDOO\ HIILFLHQW IRU DQ 0$ SURFHVV ,W LV NQRZQ WKDW WKH VDPSOH FRYDULDQFH VHTXHQFH Uf U1 fÂ§ f ` DQG WKH GDWD SHULRGRJUDP FRQVWLWXWH D )RXULHU WUDQVIRUP SDLU ,W LV DOVR NQRZQ WKDW WKH SHULRGRJUDP LV D VWDWLVWLFDOO\ LQHIILFLHQW LQ IDFW LQFRQVLVWHQWf HVWLPDWRU RI WKH 36' >@ 7KLV REVHUYDWLRQ VXJJHVWV WKDW EHWWHU FRYDULDQFH HVWLPDWRUV PLJKW EH REWDLQHG E\ )RXULHU LQYHUWLQJ EHWWHU 36' HVWLPDWRUV +RZHYHU WKLV LV QRW QHFHVVDULO\ VR %ULHIO\ VWDWHG WKH UHDVRQ LV WKDW WKH )RXULHU WUDQVIRUP DQG WKH LQYHUVH )RXULHU WUDQVIRUP DUH LQWHJUDO WUDQVIRUPV DQG KHQFH VPDOO HUURUV LQ RQH GRPDLQ PD\ EH DVVRFLDWHG ZLWK ODUJH HUURUV LQ WKH RWKHU GRPDLQ RU YLFH YHUVD 7KH IDFW WKDW WKH FRYDULDQFH HVWLPDWHV LQ f DUH FRQVLVWHQW ZKHUHDV WKH SHULRGRJUDP LV QRW LOOXVWUDWHV WKLV REVHUYDWLRQ $OVR QRWH WKDW ZKLOH WKH %XUJ HVWLPDWH RI WKH 36' LV W\SLFDOO\ PRUH DFFXUDWH WKDQ WKH SHULRGRJUDP WKH FRUUHVSRQGLQJ %XUJ HVWLPDWH RI WKH FRYDULDQFH VHTXHQFH LV JHQHUDOO\ SRRUHU WKDQ f >@ ,Q VSLWH RI WKH IDFW EULHIO\ GLVFXVVHG DERYH LQ WKH IROORZLQJ ZH FRQVLGHU HVn WLPDWLQJ WKH FRYDULDQFH VHTXHQFHV E\ LQYHUWLQJ D 36' HVWLPDWH WKDW LV RIWHQ PXFK PRUH DFFXUDWH WKDQ WKH SHULRGRJUDP QDPHO\ WKH &DSRQ 36' HVWLPDWH /LNH WKH SHn ULRGRJUDP QR PRGHO LV DVVXPHG LQ WKH &DSRQ 36' HVWLPDWRU ZKLFK PDNHV LW PRUH UREXVW WKDQ WKH SDUDPHWULF HVWLPDWRUV LQ PDQ\ VLWXDWLRQV $OWKRXJK LW KDV ORZHU VSHFWUDO UHVROXWLRQ WKDQ WKH $5 VSHFWUDO HVWLPDWRU LW JHQHUDOO\ H[KLELWV OHVV YDULDQFH PAGE 96 WKDQ WKH ODWWHU >@ >@ $QRWKHU UHDVRQ WKDW ZH FRQVLGHU WKH &DSRQ 36' HVWLPDWRU LV WKDW LW GRHV QRW H[KLELW WKH VRFDOOHG FRUUHODWLRQ PDWFKLQJ SURSHUW\ >@ >@ WKDW LV WKH LQYHUVH )RXULHU WUDQVIRUP RI WKH &DSRQ 36' HVWLPDWHV GRHV QRW \LHOG WKH VDPH FRYDULDQFH VHTXHQFHV XVHG WR REWDLQ WKH &DSRQ 36' HVWLPDWHV 7KLV IDFW DOORZV XV WR REWDLQ D QHZ FRYDULDQFH HVWLPDWRU IURP WKH &DSRQ VSHFWUD &DSRQ 36' (VWLPDWRU :H KDYH GHULYHG LQ 6HFWLRQ WKH &DSRQ ILOWHUV DQG &DSRQ DPSOLWXGH DQG SKDVH VSHFWUD ,Q WKLV VHFWLRQ ZH GHULYH H[SUHVVLRQ IRU WKH &DSRQ 36' HVWLPDWHV 7KH &DSRQ ILOWHU LV UHZULWWHQ EHORZ IRU HDV\ UHIHUHQFH VHH ff 5D0Zf A &DSRQ $F7KH ILOWHU RXWSXW SRZHU LV JLYHQ E\ e^_KI\cf_` K5K f f DAZf5D0X!ff /HW c GHQRWH WKH EDQGZLGWK RI WKH ILOWHU JLYHQ E\ f 7KHQ WKH &DSRQ 36' HVWLPDWH KDV WKH IRUP LU I?X+Q:? L X!f ^LK-\:O` f 3 DAFXf5D0Xfn 6LQFH WKH HTXLYDOHQWf WLPHEDQGZLGWK SURGXFW LV HTXDO WR XQLW\ RQH ZD\ LV WR FKRRVH 3 DV WKH UHFLSURFDO RI WKH WHPSRUDO OHQJWK RI WKH &DSRQ ILOWHU WKDW LV 0n f %\ FKRRVLQJ WKH ILOWHU EDQGZLGWK DV JLYHQ E\ f ZH REWDLQ WKH VRFDOOHG &DSRQ 36' HVWLPDWRU >@ >@ 0 &DSRQ c!Xf fÂ§ f D FXf5D0Df ZKHUH ZH KDYH UHSODFHG 5 E\ WKH VDPSOH FRYDULDQFH PDWUL[ 5 :H PD\ XVH WKH IRUZDUGRQO\ VDPSOH FRYDULDQFH PDWUL[ EXW ZH SUHIHU XVLQJ WKH IRUZDUGEDFNZDUG PAGE 97 VDPSOH FRYDULDQFH PDWUL[ JLYHQ LQ f IRU EHWWHU VWDWLVWLFDO SURSHUWLHV DVVRFLDWHV ZLWK WKH ODWWHU >@ $QRWKHU PRUH HODERUDWH FKRLFH RI LV REWDLQHG DV WKH HTXLYDOHQW EDQGZLGWK RI ?+X!f? ZKHUH +X!f LV WKH ILOWHUfV IUHTXHQF\ UHVSRQVH O K +Zf >KA}@ D 0Zf f 7KLV VSHFLILF EDQGZLGWK FKRLFH OHDGV WR WKH &DSRQ 36' HVWLPDWRU >@ >@ DtZf5D0Zf &DSRQ f DAR!f5D 0^Xf %XUJ VKRZHG WKDW WKH LQYHUVH RI &DSRQ VSHFWUXP LV HTXDO WR WKH DYHUDJH RI WKH LQYHUVHV RI WKH HVWLPDWHG $5 VSHFWUD RI RUGHUV IURP WR 0 >@ 7KLV REVHUYDWLRQ UHYHDOV WKH IDFW WKDW &DSRQ KDV OHVV VWDWLVWLFDO YDULDWLRQ DV ZHOO DV ORZHU VSHFWUDO UHVROXWLRQ WKDQ WKH $5 HVWLPDWRU $ VLPLODU EXW PRUH LQYROYHG UHODWLRQVKLS EHWZHHQ &DSRQ DQG WKH $5 HVWLPDWRUV ZDV GHULYHG LQ >@ 7KHRUHWLFDOO\ WKH SHUIRUPDQFH RI &DSRQ LV KDUG WR TXDQWLI\ +RZHYHU LW LV JHQHUDOO\ EHOLHYHG WKDW &DSRQ SRVVHVVHV ILQHU UHVROXWLRQ DQG KHQFH LV D EHWWHU VSHFWUDO HVWLPDWRU WKDQ &DSRQ >@ :H ZLOO VKRZ KHUH ZLWK D W\SLFDO H[DPSOH WKDW HYHQ WKRXJK &DSRQ KDV EHWWHU UHVROXWLRQ ORFDOO\ DURXQG WKH SRZHU SHDNV LW LV JOREDOO\ D PRUH ELDVHG HVWLPDWRU WKDQ &DSRQ 2XU H[SHULHQFH DOVR VKRZV WKDW &DSRQ JHQHUDOO\ JLYHV PXFK SRRUHU FRYDULDQFH HVWLPDWHV WKDQ &DSRQ 7KHUHIRUH &DSRQ LV QRW UHFRPPHQGHG IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ 7R LOOXVWUDWH WKH DERYH FODLP FRQVLGHU DQ $50$f VLJQDO \Qf \Q f \Q f \Q f \Q f HQf fÂ§ HQ fÂ§ f HQ fÂ§ f f ZKHUH HQf LV D UHDO ZKLWH *DXVVLDQ UDQGRP SURFHVV ZLWK ]HURPHDQ DQG XQLW YDULDQFH 7KH &DSRQ DQG &DSRQ VSHFWUDO HVWLPDWHV ZLWK 1 DQG 0 DUH VKRZQ LQ )LJXUH Df ZKHUH WKH GDVKHG FXUYH VWDQGV IRU WKH WUXH 36' RI WKH $50$ PAGE 98 $50$f1 0 UHDOL]DWLRQVf $50$f1 0 UHDOL]DWLRQVf Df Ef )LJXUH 3RZHU VSHFWUDO GHQVLW\ HVWLPDWHV E\ XVLQJ &DSRQ DQG &DSRQ 7KH SORWV DUH WKH DYHUDJHV RI LQGHSHQGHQW UHDOL]DWLRQV Df 1 0 Ef L9 0 VLJQDO ZKLOH WKH VROLG DQG WKH GDVKGRWWHG OLQHV UHVSHFWLYHO\ LQGLFDWH WKH &DSRQ DQG &DSRQ HVWLPDWHV ,W LV REYLRXV WKDW &DSRQ LV KLJKO\ ELDVHG DQG VXIIHUV IURP D VLJQLILFDQW SRZHU ORVV 7KLV REVHUYDWLRQ KDV DOVR EHHQ PDGH ZLWK RWKHU VLJQDOV HVSHFLDOO\ LI WKH VLJQDO LV D QDUURZEDQG VLJQDO $Q H[SODQDWLRQ RI WKLV EHKDYLRU IROORZV 7KH FDOFXODWLRQ RI WKH ILOWHU EDQGZLGWK LQ &DSRQ LV DSSOLFDEOH RQO\ LI WKH &DSRQ ILOWHU LV D QDUURZEDQG ILOWHU 5HFDOO WKDW WKH &DSRQ PHWKRG DLPV WR ILQG WKH &DSRQ ILOWHU WKDW PLQLPL]HV WKH WRWDO RXWSXW SRZHU RI WKH RYHUDOO IUHTXHQF\ EDQG ZKLOH SDVVHV WKH FXUUHQW IUHTXHQF\ X! XQGLVWRUWHG 1R HIIRUW KDV EHHQ WDNHQ WR PDNH VXUH WKDW WKH &DSRQ ILOWHU LV QDUURZEDQG /HW WKH OREH RI WKH &DSRQ ILOWHU IUHTXHQF\ UHVSRQVH ZKHUH WKH FXUUHQW IUHTXHQF\ RI LQWHUHVW XM LV ORFDWHG EH FDOOHG DV WKH PDLQOREH ZKLOH DOO WKH RWKHUV DUH FDOOHG WKH VLGHOREHV ,W KDV EHHQ IRXQG WKDW WKH VWHHULQJ IUHTXHQF\ XM LV QRW QHFHVVDULO\ DW WKH PD[LPXP RU WKH FHQWHU RI WKH PDLQOREH >@ >@ )XUWKHUPRUH LI WKH LQSXW VLJQDO LV D QDUURZEDQG VLJQDO WKHUH PD\ H[LVW fVLGHOREHVf ORFDWHG DW IUHTXHQF\ EDQGV ZKHUH WKH SRZHU OHYHO RI WKH LQSXW VLJQDO LV ORZ WKDW DUH HYHQ ODUJHU WKDQ WKH fPDLQOREHf RI WKH &DSRQ ILOWHU IUHTXHQF\ UHVSRQVH 1RWH PAGE 99 WKDW WKH ODUJH fVLGHOREHVf GR QRW PDNH DQ\ VLJQLILFDQW FRQWULEXWLRQV WR WKH ILOWHU RXWSXW SRZHU VR WKDW WKH ILOWHU GHVLJQ FULWHULRQ LV VWLOO VDWLVILHG WKDW LV WKH RXWSXW SRZHU LV PLQLPL]HG ZKLOH WKH IUHTXHQF\ UHVSRQVH DW WR LV RQH ,Q DOO VXFK FDVHV WKH &DSRQ ILOWHU LV QRW D QDUURZEDQG ILOWHU DQG KHQFH LW FDOFXODWHV DQ RYHUHVWLPDWHG ILOWHU EDQGZLGWK +HQFH WKH &DSRQ 36' HVWLPDWHV EHFRPH KLJKO\ ELDVHG +RZHYHU LW LV LQWHUHVWLQJ WR QRWH WKDW &DSRQ GRHV SRVVHVV KLJKHU UHVROXWLRQ FDSDELOLW\ DURXQG WKH SRZHU SHDNV WKDQ &DSRQ 7KLV LV LOOXVWUDWHG LQ )LJXUH Ef ZKLFK VKRZV WKH 36' HVWLPDWHV RI WKH VDPH $50$ VLJQDO DV XVHG LQ )LJXUH Df EXW ZLWK 1 fÂ§ DQG 0 7KH &DSRQ HVWLPDWRU FDQQRW UHVROYH WKH WZR SRZHU SHDNV WKLV WLPH ZKLOH &DSRQ DOEHLW ELDVHG VWLOO FDQ )RU WKH SUHFHGLQJ UHDVRQV ZH GR QRW FRQVLGHU XVLQJ &DSRQ IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ LQ WKH VHTXHO &DSRQ &RYDULDQFH (VWLPDWRU :H GHVFULEH EHORZ KRZ WKH &DSRQ 36' HVWLPDWHV FDQ EH )RXULHU LQYHUWHG LQ D UDWKHU FRQYHQLHQW PDQQHU \LHOGLQJ WKH &DSRQ FRYDULDQFH VHTXHQFH HVWLPDWHV 7KH VWXG\ RI WKH FRYDULDQFH VHTXHQFHV FRUUHVSRQGLQJ WR WKH &DSRQ VSHFWUD LV DQ LQWHUHVWn LQJ HQGHDYRU E\ LWVHOI ZKLFK DSSDUHQWO\ KDV QRW EHHQ XQGHUWDNHQ LQ WKH OLWHUDWXUH EHIRUH :H DOVR SUHVHQW DQ DSSUR[LPDWH EXW FRPSXWDWLRQDOO\ PRUH HIILFLHQW PHWKRG WR FDOFXODWH WKH &DSRQ FRYDULDQFH HVWLPDWHV IURP WKH &DSRQ 36' HVWLPDWHV ([DFW 0HWKRG 7KHRUHP /HW 7 ^UÂLM` ( F:3[:Lf DQG OHW G0 Zf HMXWKHQ 0 fÂ§ O DMAZfUD0Zf < KVH-68-L V 0f f f PAGE 100 ZKHUH PLQ0VfÂ§0fÂ§f $ fÂ§ A U NNV f IH PD[Vf ,I7 LV +HUPLWLDQ WKHQ ÂLV MLrBV 3URRI 6HH $SSHQGL[ ,W LV REYLRXV WKDW ZKHQHYHU 7 LV QRQQHJDWLYH GHILQLWH ZH KDYH DMARfUD0Zf IRU DQ\ WR 7KXV DAZfUD0LMf LV D YDOLG SRZHU VSHFWUXP )XUWKHUPRUH 7KHRUHP LQGLFDWHV WKDW LQ VXFK D FDVH ODMAXfUD$FMf LV LQ IDFW WKH SRZHU VSHFWUXP RI DQ 0 fÂ§ OfWKRUGHU $5 SURFHVV &RQVHTXHQWO\ &DSRQ \LHOGV DQ HTXLYDOHQW $50 fÂ§ f SURFHVV ZKHUHDV &DSRQ \LHOGV DQ HTXLYDOHQW $50$0 fÂ§ 0 fÂ§ f SURFHVVf %\ PDNLQJ XVH RI f ZH FDQ ILQG WKH FRHIILFLHQWV RI WKH HTXLYDOHQW $5 SURFHVV 7KH FDOFXODWLRQ RI WKH H[DFW FRYDULDQFH VHTXHQFHV IURP WKH $5 FRHIILFLHQWV LV D VWDQGDUG SUREOHP DQG FDQ EH VROYHG IRU H[DPSOH YLD WKH LQYHUVH /HYLQVRQ'XUELQ DOJRULWKP 6HH >@ >@ DQG WKH UHIHUHQFHV WKHUHLQ IRU PRUH GHWDLOVf +HQFH WKH LPSOHPHQWDWLRQ RI WKH &DSRQ PHWKRG IRU FRYDULDQFH HVWLPDWLRQ UXQV DV RXWOLQHG EHORZ 6WHS 3LFN XS D YDOXH IRU 0^0 1f DQG FRPSXWH 5 E\ f 6WHS &RPSXWH QV DVVRFLDWHG ZLWK 7 5U E\ f )DFWRUL]H 0fÂ§O P fÂ§0fÂ§f VD\ E\ XVLQJ WKH 1HZWRQ5DSKVRQ DOJRULWKPf DQG REWDLQ WKH PLQLPXPSKDVHf VSHFWUDO IDFWRU 6WHS &RPSXWH WKH FRUUHVSRQGLQJ FRYDULDQFH VHTXHQFH IWHPSAf` IURP WKH VSHFWUDO IDFWRU RU HTXLYDOHQWO\ WKH $5 PRGHOf E\ IRU H[DPSOH WKH LQYHUVH /HYLQVRQ'XUELQ DOJRULWKP :KLOH XVLQJ WKH )RXULHU LQYHUWLQJ PHWKRG IRU FRYDULDQFH HVWLPDWLRQ LW LV QHFn HVVDU\ WKDW WKH LQWHJUDO RI WKH 36' HVWLPDWH RYHU DOO IUHTXHQFLHV JLYHV D JRRG HVWLPDWH RI WKH VLJQDO SRZHU RWKHUZLVH WKHUH PD\ EH VFDOLQJ HUURUV LQ WKH FRYDULDQFH HVWLPDWHV PAGE 101 6R DIWHU 6WHS ZH XVH D SRZHU FRPSHQVDWLRQ DSSURDFK WR REWDLQ RXU XOWLPDWH &DSRQ FRYDULDQFH HVWLPDWHV 2XU ILQDO HVWLPDWHV ^A&DSRQLAf` KDYH WKH IRUP Q ? B If AWHPSAf U&f3rQ:f39H}Sff ZKHUH 3 LV WKH HVWLPDWHG SRZHU RI WKH VLJQDO Â‘ Q f f $SSUR[LPDWH 0HWKRG 6LQFH WKH &DSRQ VSHFWUXP LV HTXLYDOHQW WR DQ $5 VSHFWUXP WKH FRYDULDQFH VHTXHQFH FDQ EH FRPSXWHG H[DFWO\ DV GHVFULEHG DERYH :LWK VRPH DFFXUDF\ ORVV WKH FRPSXWDWLRQDO GHPDQG RI WKH &DSRQ PHWKRG FDQ EH UHGXFHG $FFRUGLQJ WR WKH :LHQHU.KLQWFKLQH WKHRUHP ZH KDYH UNf f :H FDQ UHZULWH f DV UQ UNf fÂ§ fÂ§ Wf^XfHONXGX f -R /HW Â“ 1 DQG OHW WKH &DSRQ VSHFWUXP EH HYDOXDWHG DW X! LUÂ“? WKDW LV ZH FDOFXODWH !R!f Â“ fÂ§ 7KHQ ZH FDQ DSSUR[LPDWH WKH FRPSXWDWLRQ RI WKH FRYDULDQFH VHTXHQFH FRUUHVSRQGLQJ WR !XMf E\ WKH IROORZLQJ HTXDWLRQ INf WW W ,9 1fÂ§O 1 1 M!XQfHLN8 f ZKLFK FDQ EH HYDOXDWHG E\ XVLQJ ))7 6LQFH WKH HUURU LQ DSSUR[LPDWLQJ WKH LQWHJUDn WLRQ E\ WKH VXPPDWLRQ DERYH LV OL9f WKH HUURUV LQWURGXFHG DUH TXLWH VPDOO IRU ODUJH HQRXJK Â“ 2XU FRPSXWHU VLPXODWLRQV DOVR FRQILUP WKLV REVHUYDWLRQ 7KH HYDOXDWLRQ RI IfXMf LQ f E\ GLUHFWO\ XVLQJ f LV FRPSXWDWLRQDOO\ EXUGHQVRPH :H FDQ LQVWHDG PDNH XVH RI f DV IROORZV 6LQFH 5B LV +HUPLWLDQ PAGE 102 E\ 7KHRUHP ZH KDYH cLV cLrBV 7KHQ 0fÂ§ 0IAf < AH-68- V fÂ§0fÂ§f 0 ( ( IWf 0fÂ§ V V fÂ§0fÂ§f 0O $I ( A ÂBVHAr LR -6L2IL V A:f N!ILf fÂ§ $Â f ZKHUH 9 f V ZKLFK DJDLQ FDQ EH HYDOXDWHG E\ XVLQJ ))7 ZLWK ]HURSDGGLQJ LH IRU V 0 Â“ &RPSXWDWLRQDO $VSHFWV :H EULHIO\ GLVFXVV WKH FRPSXWDWLRQDO DVSHFWV RI WKH VWDQGDUG PHWKRG DQG WKH &DSRQ PHWKRG IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ 5HFDOO WKDW 0 DQG 1 GHQRWH WKH ODUJHVW ODJ RI WKH FRYDULDQFH HVWLPDWHV WKH OHQJWK RI WKH &DSRQ ILOWHU DQG WKH QXPEHU RI GDWD VDPSOHV UHVSHFWLYHO\ $VVXPH WKDW WKH GDWD LV UHDO 1 1 0 DQG 0 7KHQ WKH VWDQGDUG PHWKRG LQYROYHV DSSUR[LPDWHO\ .1 IORSV 7KH QXPEHU RI IORSV QHHGHG E\ WKH H[DFW &DSRQ PHWKRG LV GLIILFXOW WR GHWHUn PLQH )RU D PRGHUDWH 0 VD\ 1 0 1 ZH ILQG WKDW WKH FRPSXWDWLRQDOO\ PRVW GHPDQGLQJ SDUW RI WKH H[DFW &DSRQ PHWKRG LV WKH VSHFWUDO IDFWRUL]DWLRQ 6LQFH WKH DOJRULWKP 1HZWRQ5DSKVRQf XVHG WR ILQG WKH PLQLPXPSKDVH VSHFWUDO IDFWRU LV LWHUDWLYH WKH QXPEHU RI IORSV QHHGHG LV KDUG WR TXDQWLI\ $QRWKHU VLJQLILFDQW FRPSXn WDWLRQDO GHPDQG LV WKH FDOFXODWLRQ RI WKH VDPSOH FRYDULDQFH PDWUL[ 5 ZKLFK LQYROYHV DSSUR[LPDWHO\ 10 0 IORSV +RZHYHU ZH QRWH WKDW WKH DPRXQW RI FRPSXWDn WLRQV QHHGHG E\ WKH IDFWRUL]DWLRQ LV PXFK ODUJHU WKDQ WKDW QHHGHG IRU FRPSXWLQJ WKH VDPSOH FRYDULDQFH PDWUL[ IRU D PRGHUDWH RU ODUJH 0 PAGE 103 )ORS 5DWLR e ([DFW &DSRQ $SSUR[LPDWH &DSRQ 0 0 ,, ,, Dr 7DEOH &RPSDULVRQ RI WKH FRPSXWDWLRQDO EXUGHQV RI WKH VWDQGDUG DQG &DSRQ PHWKRGV ZLWK 1 DQG Â“ 7KH DSSUR[LPDWH &DSRQ PHWKRG QR ORQJHU QHHGV WR SHUIRUP WKH VSHFWUDO IDFn WRUL]DWLRQ DQG WKH FRPSXWDWLRQDO GHPDQG LV JUHDWO\ UHGXFHG 7KH DSSUR[LPDWH &DSRQ PHWKRG UHTXLUHV DSSUR[LPDWHO\ 10 0 IORSV D ODUJH SDUW RI ZKLFK FRPHV IURP WKH FRPSXWDWLRQ RI WKH VDPSOH FRYDULDQFH PDWUL[ 5 7R LOOXVWUDWH TXDQWLWDWLYHO\ WKH FRPSXWDWLRQDO EXUGHQV RI WKH DERYH WKUHH PHWKRGV ZH VKRZ D IHZ VLPXODWLRQ UHVXOWV LQ 7DEOH ZKHUH ZH GHILQH e DV WKH UDWLR RI WKH IORSV QHHGHG E\ WKH H[DFW RU DSSUR[LPDWH &DSRQ PHWKRG WR WKDW FRUUHn VSRQGLQJ WR WKH VWDQGDUG PHWKRG :H UHPDUN WKDW ZH GLG QRW SD\ VSHFLDO DWWHQWLRQ WR WKH FRGLQJ RI WKH DOJRULWKPV KHQFH WKH QXPEHUV SURYLGHG KHUH VKRXOG EH RQO\ WDNHQ DV LQGLFDWLYH RI WKH FRPSXWDWLRQDO FRPSOH[LWLHV RI WKH PHWKRGV 1XPHULFDO 5HVXOWV ,Q WKLV VHFWLRQ ZH SUHVHQW QXPHULFDO H[DPSOHV VKRZLQJ WKH SHUIRUPDQFH RI WKH &DSRQ PHWKRG IRU FRYDULDQFH HVWLPDWLRQ 7KH ILUVW SUREOHP DGGUHVVHG LV WKH $50$ FRYDULDQFH HVWLPDWLRQ 6HYHUDO $50$ VLJQDOV ZLWK GLIIHUHQW SROH DQG ]HUR ORFDWLRQV DUH JHQHUDWHG WR FRPSDUH WKH SHUIRUPDQFHV RI WKH VWDQGDUG DQG WKH &DSRQ PHWKRGV :H DOVR FRQVLGHU $5 FRHIILFLHQW HVWLPDWLRQ RI WKH $50$ VLJQDOV E\ XVLQJ WKH &DSRQ DQG VWDQGDUG FRYDULDQFH HVWLPDWHV )LQDOO\ ZH JLYH DQ H[DPSOH LOOXVWUDWLQJ WKH 0$ PRGHO GHWHUPLQDWLRQ ZLWK WKH VWDQGDUG DQG &DSRQ PHWKRGV PAGE 104 3URFHVVHV $50$ $50$ $50$ $50$ DL D E? E A E D 7DEOH 7KH $50$ SURFHVVHV XVHG LQ WKH QXPHULFDO VLPXODWLRQV $50$ &RYDULDQFH (VWLPDWLRQ )LUVW ZH FRPPHQW RQ WKH JHQHUDWLRQ RI WKH $50$ VLJQDOV 7R HOLPLQDWH WKH LQLWLDO WUDQVLHQW FDXVHG E\ LPSURSHU LQLWLDOL]DWLRQ RI WKH $50$ V\VWHP ZH SURFHHG DV IROORZV *LYHQ WKH FRHIILFLHQWV RI DQ $50$STf ZH FDQ GHWHUPLQH WKH FRYDULDQFH PDWUL[ VHH HJ >@f Uf UOf f f f US fÂ§ f UrOf Uf UOf UrS fÂ§ f f f Â‘ UrOf Uf f :H XVH &H DV WKH LQLWLDO FRQGLWLRQ IRU WKH $50$ V\VWHP ZKHUH H &S[O LV D ]HURPHDQ *DXVVLDQ UDQGRP YHFWRU ZLWK LGHQWLW\ FRYDULDQFH PDWUL[ )RXU $50$ VLJQDOV DUH FKRVHQ IRU RXU VWXG\ ,Q WKH VHOHFWLRQ RI WKHVH $50$ VLJQDOV HIIRUWV KDYH EHHQ PDGH WR PDNH WKHP UHSUHVHQWDWLYH RI D ODUJH FODVV RI $50$ VLJQDOV 7KH FRHIILFLHQWV RI WKH $50$ VLJQDOV DUH OLVWHG LQ 7DEOHV 7KH SROH DQG ]HUR GLDJUDPV DUH VKRZQ LQ )LJXUHV Df WR Gf )LJXUHV Df WR Gf JLYH WKH 36'fV RI WKH $50$ VLJQDOV ZKLOH )LJXUHV Df WR Gf VKRZ WKH FRUUHVSRQGLQJ FRYDULDQFH SORWV IRU WKHVH VLJQDOV PAGE 105 Ff Gf )LJXUH 3ROH]HUR GLDJUDPV IRU $50$ WHVW FDVHV Df $50$ Ef $50$ Ff $50 $ Gf $50$ ,Q WKH IROORZLQJ QXPHULFDO H[DPSOHV ZH VHW 1 fÂ§ VKRUWHU GDWD OHQJWKV KDYH DOVR EHHQ FRQVLGHUHG DQG WKH UHVXOWV DUH VLPLODUf DQG 0 fÂ§ VHH 6HFWLRQV DQG IRU KRZ WR FKRRVH 0f 7KH SHUIRUPDQFHV RI WKH VWDQGDUG DQG WKH &DSRQ PHWKRGV IRU HDFK RI WKH IRXU $50$ PRGHOV DUH VKRZQ LQ )LJXUHV Df WR Gf ZKHUH WKH FXUYHV VKRZ WKH PHDQVTXDUHG HUURUV 06(f RI WKH FRYDULDQFH HVWLPDWHV QRUPDOL]HG ZLWK UHVSHFW WR Uf YHUVXV WKH WLPHODJ RI WKH FRYDULDQFH VHTXHQFHV 2QO\ WKH UHVXOWV IURP WKH H[DFW &DSRQ PHWKRG DUH GHPRQVWUDWHG RZQLQJ WR VSDFH OLPLWDWLRQf 7KH 06( YDOXHV DUH EDVHG RQ LQGHSHQGHQW UHDOL]DWLRQV ,W KDV EHHQ IRXQG WKDW WKH &DSRQ PHWKRG JHQHUDOO\ JLYHV EHWWHU UHVXOWV WKDQ WKH VWDQGDUG PHWKRG HVSHFLDOO\ IRU ODUJH WLPHODJV :H DOVR QRWH WKDW ZKHQ WKH SROHV DUH FORVH WR WKH XQLW FLUFOH DV LQ )LJXUH Ef WKH SHUIRUPDQFH GLIIHUHQFHV EHWZHHQ WKH WZR PHWKRGV DUH QRW VR ODUJH DV LQ WKH RWKHU FDVHV 7KRXJK WKH &DSRQ PHWKRG LV VWLOO EHWWHU WKDQ WKH VWDQGDUG PHWKRGf PAGE 106 3RZHU 6SHFWUDO 'HQVLW\G% 3RZHU 6SHFWUDO 'HQVLW\G% $50$ $50$ Df $50$ &f Ef $50$ Gf )LJXUH 7UXH SRZHU VSHFWUDO GHQVLWLHV Df $50$ Ef $50$ Ff $50$ Gf $50$ PAGE 107 )LJXUH 7UXH FRYDULDQFH VHTXHQFHV Df $50$ Ef $50$ Ff $50$ &RYDULDQFH 6HTXHQFHV Â S I O O OfÂ§ &RYDULDQFH 6HTXHQFHV 4 4 $ B 2 $50$ $50$ &RYDULDQFH 6HTXHQFHV &' &Q PAGE 108 $50$ 1 0 UHDOL]DWLRQVf $50$ 1 0 UHDOL]DWLRQVf Df Ef $50$ 1 0 UHDOL]DWLRQVf $50$ 1 0 UHDOL]DWLRQVf Ff Gf )LJXUH &RYDULDQFH VHTXHQFH HVWLPDWLRQ ZLWK 1 DQG 0 7KH PHDQ VTXDUHG HUURUV 06(Vf RI WKH FRYDULDQFH HVWLPDWHV QRUPDOL]HG ZLWK UHVSHFW WR Uf DUH EDVHG RQ LQGHSHQGHQW UHDOL]DWLRQV Df $50$ Ef $50$ Ff $50 $ Gf $50$ PAGE 109 $5 &RHIILFLHQW (VWLPDWLRQ IRU $50$ 6LJQDOV $V DQ DSSOLFDWLRQ RI WKH &DSRQ PHWKRG IRU FRYDULDQFH HVWLPDWLRQ ZH LQFOXGH KHUH DQ H[DPSOH RQ KRZ WR XVH WKH &DSRQ FRYDULDQFH HVWLPDWHV WR ILQG WKH $5 FRHIILFLHQWV RI $50$ VLJQDOV YLD WKH RYHUGHWHUPLQHG PRGLILHG PAGE 110 7KLV LV HVSHFLDOO\ WKH FDVH LI WKHUH DUH SROHV LQ WKH $50$ PRGHO FORVH WR WKH XQLW FLUFOH VLQFH WKH FRYDULDQFH VHTXHQFH GHFD\V WR ]HUR YHU\ VORZO\ LQ VXFK D FDVH 7KH SHUIRUPDQFHV RI WKH $5 FRHIILFLHQW HVWLPDWLRQ IRU WKH $50$ VLJQDOV YLD WKH 20<: PHWKRG E\ XVLQJ ERWK WKH VWDQGDUG DQG &DSRQ FRYDULDQFH HVWLPDWHV DUH VKRZQ LQ )LJXUHV Df WR Gf ZKHUH 1 DQG 0 DQG ZKHUH WKH SORWV DUH DJDLQ EDVHG RQ LQGHSHQGHQW WULDOV 7KH 20<: HTXDWLRQV DUH VROYHG E\ XVLQJ WKH /6 PHWKRG 7R UHGXFH WKH QXPEHU RI ILJXUHV WKH FXUYHV DUH WKH VXP RI WKH 06(fV RI DOO $5 FRHIILFLHQW HVWLPDWHV YHUVXV WKH QXPEHU RI HTXDWLRQV XVHG LQ WKH 20<: PHWKRG :KHQ XVLQJ WKH 20<: PHWKRG ZH DVVXPH WKDW WKH $5 RUGHUV DUH NQRZQ 7KH QXPEHUV RI HTXDWLRQV XVHG LQ WKH FXUYHV DUH / DQG ,W KDV EHHQ IRXQG WKDW WKH $5 FRHIILFLHQW HVWLPDWHV REWDLQHG E\ XVLQJ WKH &DSRQ FRYDULDQFH HVWLPDWHV DUH XVXDOO\ EHWWHU WKDQ WKRVH REWDLQHG E\ WKH VDPSOH FRYDULDQFH HVWLPDWHV 1RWH WKDW IRU WKH 20<: HVWLPDWRU EDVHG RQ WKH VDPSOH FRYDULDQFH HVWLPDWHV WKH HVWLPDWLRQ SHUIRUPDQFH PD\ VLJQLILFDQWO\ GHWHULRUDWH ZLWK LQFUHDVLQJ / D ODUJH YDOXH RI / PD\ EH XVHG IRU ODFN RI D SULRUL LQIRUPDWLRQ RQ WKH $50$ VLJQDO LQ TXHVWLRQf ZKHUHDV WKH &DSRQ20<: HVWLPDWRUfV SHUIRUPDQFH LV PXFK OHVV DIIHFWHG E\ WKH LQFUHDVH RI / 0$ 0RGHO 2UGHU 'HWHUPLQDWLRQ )RU DQ 0$Jf SURFHVV WKH FRYDULDQFHV ZLWK ODJV ODUJHU WKDQ T DUH DOO ]HURV >@ >@ +RZHYHU ZH FDQ H[SHFW WKDW WKH VDPSOH FRYDULDQFH HVWLPDWHV IRU DQ 0$ VLJQDO ZLOO QRW GHFD\ WR ]HUR IDVW HQRXJK ZKLOH WKH &DSRQ FRYDULDQFH HVWLPDWHV ZLOO XVXDOO\ JLYH INf P IRU N T VLQFH WKH &DSRQ PHWKRG JLYHV EHWWHU FRYDULDQFH HVWLPDWHV IRU KLJKHU ODJV +HQFH WKH LQIHUHQFH DERXW WKH W\SH RI VLJQDO ZH DUH GHDOLQJ ZLWK DQG LWV RUGHU ZLOO EH HDVLHU WR PDNH ZLWK WKH &DSRQ PHWKRG :H FRQVLGHU DQ 0$f VLJQDO \^Qf HQf HQ f HQ f HQ f HQ f f PAGE 111 $50$ 1 0 UHDOL]DWLRQVf $50$ 1 0 UHDOL]DWLRQVf Df Ef Ff Gf )LJXUH 7KH $5 FRHIILFLHQW HVWLPDWLRQ RI WKH $50$ VLJQDOV YLD WKH RYHUGHWHUn PLQHG PRGLILHG PAGE 112 Df Ef )LJXUH VXSHULPSRVHG UHDOL]DWLRQV RI WKH 0$ FRYDULDQFH VHTXHQFH HVWLPDWHV ZLWK 1 DQG 0 Df 7KH VWDQGDUG PHWKRG Ef 7KH &DSRQ PHWKRG :H XVH ERWK WKH VWDQGDUG DQG WKH &DSRQ PHWKRGV WR GHWHUPLQH LWV FRYDULDQFH HVWLn PDWHV 7KH UHVXOWV DUH VKRZQ LQ )LJXUHV Df DQG Ef ZKHUH 1 fÂ§ DQG 0 DQG ZKHUH VXSHULPSRVHG UHDOL]DWLRQV IRU ERWK PHWKRGV DUH GLVSOD\HG $V H[SHFWHG WKH &DSRQ FRYDULDQFH HVWLPDWHV GHFD\ WR DOPRVW ]HUR DIWHU N ZKLOH WKH VWDQGDUG FRYDULDQFH HVWLPDWHV DUH PXFK PRUH HUUDWLF &RQVHTXHQWO\ WKH LQIHUHQFH WKDW WKH SURFHVV XQGHU VWXG\ LV DQ 0$f LV HDVLHU WR PDNH E\ XVLQJ WKH &DSRQ FRYDULDQFH HVWLPDWRU 6XPPDU\ 7KH &DSRQ PHWKRG IRU FRYDULDQFH VHTXHQFH HVWLPDWLRQ PDNHV XVH RI )RXULHU LQYHUVLRQ RI WKH &DSRQ VSHFWUDO HVWLPDWHV ,W LV VKRZQ WKDW WKH &DSRQ FRYDULDQFH HVWLPDWHV DUH XVXDOO\ PRUH DFFXUDWH WKDQ WKH VWDQGDUG ELDVHG VDPSOH FRYDULDQFH HVWLPDWHV HVSHFLDOO\ IRU ODUJH ODJV 7KH &DSRQ VSHFWUXP LV HTXLYDOHQW WR DQ $5 VSHFWUXP DQG KHQFH WKH FRUUHVSRQGLQJ FRYDULDQFH VHTXHQFH FDQ EH H[DFWO\ FDOFXODWHG YLD WKH LQYHUVH /HYLQVRQ'XUELQ DOJRULWKP 6LQFH WKH VSHFWUDO IDFWRUL]DWLRQ QHHGHG IRU WKH H[DFW &DSRQ FRYDULDQFH HVWLPDWLRQ LV LQ JHQHUDO FRPSXWDWLRQDOO\ H[SHQVLYH PAGE 113 WKH DSSUR[LPDWH &DSRQ PHWKRG ZKLFK DYRLGV WKH VSHFWUDO IDFWRUL]DWLRQ DQG PDNHV XVH RI ))7 PD\ EH SUHIHUUHG LQ SUDFWLFDO DSSOLFDWLRQ PAGE 114 &+$37(5 &21&/86,216 6XPPDU\ 5HPDUNV ,Q WKLV ZRUN ZH KDYH GLVFXVVHG XVLQJ 0$WFKHG),OWHUEDQN 0$),f DSSURDFKHV IRU VSHFWUDO DQDO\VLV DQG SDUDPHWHU HVWLPDWLRQ 6SHFLILFDOO\ WKH SUREOHPV RI FRPSOH[ VSHFWUDO HVWLPDWLRQ VLQXVRLGDO DPSOLWXGH HVWLPDWLRQ DQG FRYDULDQFH VHTXHQFH HVWLn PDWLRQ KDYH EHHQ DGGUHVVHG ,Q &KDSWHU ZH ILUVW LQWURGXFHG WKH 0$), FRQFHSW ZKLFK LV RI SULPDU\ LPSRUWDQFH WR WKLV VWXG\ E\ FKRRVLQJ WKH ILOWHUEDQN DV PDWFKHGILOWHUEDQN LH HDFK ILOWHU LQ WKH ILOWHUEDQN LV D PDWFKHG ILOWHU 6LQFH ZH GR QRW PRGHO WKH REVHUYDWLRQ QRLVH H[DFWO\ WKHUH DUH YDULRXV ZD\V RI HVWLPDWLQJ WKH FRYDULDQFH PDWUL[ RI WKH REVHUYDWLRQ QRLVH ZKLFK OHDG WR GLIIHUHQW 0$), HVWLPDWRUV IRU VSHFLILF HVWLPDWLRQ SXUSRVHV )RU FRPSOH[ VSHFWUDO HVWLPDWLRQ LW ZDV VKRZQ LQ &KDSWHU WKDW WKH &DSRQ DQG $3(6 HVWLPDWRUV ZKLOH RULJLQDOO\ QRW GHULYHG LQ WKH 0$), IUDPHZRUN DUH ERWK PHPEHUV RI WKH 0$), FRPSOH[ VSHFWUDO HVWLPDWRUV 0RUHRYHU WKH 0$), FRQFHSW PD\ EH XVHG WR GHYLVH RWKHU LQWHUHVWLQJ VSHFWUDO HVWLPDWRUV $OWKRXJK &DSRQ DQG $3(6 DUH ERWK DV\PSWRWLFDOO\ HIILFLHQW DQG DUH RI VLPLODU FRPSXWDWLRQDO FRPSOH[LW\ WKHLU SHUIRUPDQFHV DUH TXLWH GLIIHUHQW IRU ILQLWH OHQJWK RI GDWD VDPSOHV 6SHFLILFDOO\ ZH SURYHG WKDW WR ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQ &DSRQ XQGHUHVWLPDWHV WKH WUXH VSHFWUXP ZKHUHDV $3(6 LV XQELDVHG DQG WKDW WKH ELDV RI WKH IRUZDUGEDFNZDUG &DSRQ LV RQH KDOI WKDW RI WKH IRUZDUGRQO\ &DSRQ 7KHVH UHVXOWV DUH EHOLHYHG WR SURn YLGH FRPSHOOLQJ UHDVRQV IRU SUHIHUULQJ $3(6 WR &DSRQ LQ PRVW SUDFWLFDO DSSOLFDWLRQV ,W VKRXOG EH VWUHVVHG WKDW WKH EHWWHU HVWLPDWLRQ DFFXUDF\ REWDLQHG E\ &DSRQ RU $3(6 RYHU WKH )RXULHUEDVHG PHWKRGV LV DFKLHYHG DW WKH FRVW RI FRPSXWDWLRQDO PAGE 115 FRPSOH[LW\ 7KLV EHFRPHV PRUH HYLGHQW LQ VSHFWUDO HVWLPDWLRQ ZKHUH WKH DPRXQW RI FRPSXWDWLRQ UHTXLUHG E\ HLWKHU &DSRQ RU $3(6 LV PXFK PRUH WKDQ E\ WKH )RXULHU EDVHG PHWKRGV (IILFLHQW LPSOHPHQWDWLRQ VFKHPHV ZHUH SUHVHQWHG LQ &KDSWHU IRU ERWK WKH &DSRQ DQG $3(6 HVWLPDWRUV ,W ZDV VKRZQ WKDW WKH HIILFLHQW LPSOHPHQWDn WLRQ VFKHPH RI &DSRQ RU $3(6 UHGXFHV WKH FRPSXWDWLRQDO FRPSOH[LW\ VXEVWDQWLDOO\ HVSHFLDOO\ ZKHQ WKH QXPEHU RI IUHTXHQF\ VDPSOHV DUH ODUJH ,Q FRPSOH[ VSHFWUDO HVWLPDWLRQ WKH ORFDWLRQV RI WKH VSHFWUDO FRPSRQHQWV WKDW DUH RI LQWHUHVW WR XV DUH LQ JHQHUDO XQNQRZQ +HQFH WKH FHQWHU IUHTXHQF\ RI WKH SUHILOWHU PXVW VZHHS WKURXJK DOO IUHTXHQF\ EDQGV WR REWDLQ WKH FRUUHVSRQGLQJ VSHFn WUDO HVWLPDWHV 2QH SUREOHP WKDW LV FORVHO\ UHODWHG WR FRPSOH[ VSHFWUDO HVWLPDWLRQ LV DPSOLWXGH HVWLPDWLRQ IRU VLQXVRLGDO VLJQDOV ZLWK NQRZQ IUHTXHQFLHV &RQYHQWLRQDO PHWKRGV IRU DPSOLWXGH HVWLPDWLRQ DUH EDVHG RQ WKH WHFKQLTXH RI /HDVW 6TXDUHV /6f ILWWLQJ +RZHYHU /6 PHWKRGV W\SLFDOO\ SHUIRUP TXLWH SRRUO\ ZKHQHYHU WKH REVHUYDn WLRQ QRLVH LV FRORUHG %\ VSOLWWLQJ WKH GDWD YHFWRUV LQWR VXEYHFWRUV RI VKRUWHU OHQJWK :HLJKWHG /HDVW 6TXDUHV :/6f WHFKQLTXH FDQ EH XVHG IRU DPSOLWXGH HVWLPDWLRQ :H SUHVHQWHG LQ &KDSWHU VHYHUDO :/6 DPSOLWXGH HVWLPDWRUV DQG LQWHUHVWLQJO\ HQRXJK WKH\ DUH FORVHO\ UHODWHG WR WKH &DSRQ DQG $3(6 HVWLPDWRUV IRU FRPSOH[ VSHFWUDO HVn WLPDWLRQ :H DOVR SURSRVHG D JHQHUDOL]HG 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ ZKHUH DOO SUHILOWHUV FRUUHVSRQGLQJ WR WKH GLIIHUHQW IUHTXHQFLHV DUH VLPXOWDQHRXVO\ GHn WHUPLQHG E\ GHILQLQJ DQG PD[LPL]LQJ D fJHQHUDOL]HG 615f ,W ZDV IRXQG WKDW WKH JHQHUDOL]HG 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ LV YHU\ JHQHUDO DQG LQFOXGHV WKH :/6 DSSURDFK DV D VSHFLDO FDVH :H GHULYHG DQ LQWHUHVWLQJ 0$), DPSOLWXGH HVWLPDn WRUV WKDW GRHV QRW IDOO LQWR WKH :/6 FODVV %XW RWKHU 0$), DPSOLWXGH HVWLPDWRUV PD\ H[LVW DQG DUH \HW WR EH GLVFRYHUHG $QRWKHU SUREOHP ZH DGGUHVVHG LV FRYDULDQFH VHTXHQFH HVWLPDWLRQ 6SHFLILFDOO\ ZH SUHVHQWHG LQ &KDSWHU D QHZ FRYDULDQFH VHTXHQFH HVWLPDWRU E\ )RXULHU LQYHUWLQJ WKH &DSRQ 3RZHU 6SHFWUDO 'HQVLW\ 36'f HVWLPDWRU :H GHVFULEHG KRZ WR )RXULHU PAGE 116 LQYHUWLQJ WKH &DSRQ 36' HVWLPDWHV LQ ERWK WKH H[DFW DQG DSSUR[LPDWH PDQQHUV 7KH DSSUR[LPDWH &DSRQ FRYDULDQFH HVWLPDWHV DUH FRPSXWDWLRQDOO\ PRUH FRQYHQLHQW WR REWDLQ WKDQ WKH H[DFW &DSRQ FRYDULDQFH HVWLPDWHV DQG WKH DFFXUDF\ ORVV LV LQ JHQHUDO YHU\ VPDOO SURYLGHG WKDW VXIILFLHQW ODUJH IUHTXHQF\ VDPSOHV DUH XVHG ,W ZDV IRXQG WKDW WKH &DSRQ FRYDULDQFH VHTXHQFH HVWLPDWRU JLYHV PRUH DFFXUDWH FRYDULDQFH HVWLPDWHV WKDQ WKH ZLGHO\XVHG VDPSOH FRYDULDQFH VHTXHQFHV HVSHFLDOO\ IRU WKH KLJKHU ODJV ,Q VXPPDU\ ZH KDYH f LQWURGXFHG WKH 0$), LGHD IRU VSHFWUDO DQDO\VLV DQG SDUDPHWHU HVWLPDWLRQ f VKRZQ WKDW WKH &DSRQ DQG $3(6 FRPSOH[ HVWLPDWRUV DUH ERWK PHPEHUV RI WKH 0$), DSSURDFK WR FRPSOH[ VSHFWUDO HVWLPDWLRQ f SURYHG WKDW WKH HVWLPDWLRQ HUURUV RI WKH &DSRQ DQG $3(6 FRPSOH[ VSHFWUDO HVWLPDWRUV KDYH VLPLODU DV\PSWRWLF GLVWULEXWLRQ f SURYHG WKDW IRU ILQLWH OHQJWK RI GDWD VDPSOHV WKH &DSRQ FRPSOH[ VSHFWUDO HVWLn PDWRU LV ELDVHG GRZQZDUGf ZKHUHDV WKH $3(6 FRPSOH[ VSHFWUDO HVWLPDWRU LV XQELDVHG DQG WKDW WKH ELDV RI WKH IRUZDUGEDFNZDUG &DSRQ LV RQH KDOI WKDW RI WKH IRUZDUGRQO\ &DSRQ WR ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQf f GHYLVHG HIILFLHQW LPSOHPHQWDWLRQ VFKHPHV IRU WKH &DSRQ DQG $3(6 FRPSOH[ VSHFWUDO HVWLPDWRUV f SUHVHQWHG D JHQHUDOL]HG 0$), DSSURDFK DORQJ ZLWK VHYHUDO RWKHU LQWHUHVWLQJ PHWKRGV IRU DPSOLWXGH HVWLPDWLRQ f GHPRQVWUDWHG WKDW WKH 0$), DSSURDFK WR DPSOLWXGH HVWLPDWLRQ LQFOXGHV WKH :/6 DSSURDFK PAGE 117 f SUHVHQWHG D QHZ FRYDULDQFH VHTXHQFH HVWLPDWRU E\ )RXULHU LQYHUWLQJ WKH &DSRQ 36' HVWLPDWHV f GHVFULEHG KRZ WR )RXULHU LQYHUWLQJ WKH &DSRQ VSHFWUD H[DFWO\ ZKLFK LV DQ LQn WHUHVWLQJ HQGHDYRU E\ LWVHOI )XWXUH :RUN 2QH LQWHUHVWLQJ DUHD RI IXWXUH ZRUN LV DPSOLWXGH HVWLPDWLRQ IRU VLQXVRLGDO VLJQDOV ZLWK DSSOLFDWLRQ WR V\VWHP LGHQWLILFDWLRQf +RZHYHU XQOLNH WKH H[WHQVLRQ RI WKH 0$), FRPSOH[ VSHFWUDO HVWLPDWRUV IURP WKH FDVH WR WKH FDVH WKH H[WHQVLRQ IRU DPSOLWXGH HVWLPDWLRQ LV QRW VR VWUDLJKWIRUZDUG $ ILUVW TXHVWLRQ ZRXOG EH WKH DQDO\VLV RI /6( .f DQG /6( f LQ WKH FRORUHG QRLVH FDVH 7KHUH PD\ EH QR FRPSDFW H[SUHVVLRQ IRU WKH DV\PSWRWLF DFFXUDF\ WKDW LV VLPLODU WR WKH RQH LQ WKH FDVH 7KH UHDVRQ LV WKDW WKHUH H[LVW LQ WKH FDVH QR VXFK H[SUHVVLRQV DV WKRVH VKRZQ LQ f DQG f $QRWKHU LPSRUWDQW TXHVWLRQ ZRXOG EH KRZ WR DSSO\ :/6 WR WKH GDWD 7KLV LQYROYHV KRZ WR FUHDWH WKH VXEYHFWRUV IURP WKH GDWD PDWUL[ DQG KRZ WR HVWLPDWH WKH FRYDULDQFH PDWUL[ RI WKH QRLVH VXEYHFWRUV HWF 2QH FDQ H[SHFW WKDW WKH :/6 DPSOLWXGH HVWLPDWRUV PD\ EH FRQVLGHUDEO\ PRUH FRPSOH[ WKDQ WKHLU FRXQWHUSDUWV PAGE 118 $33(1',; $ 3522) 2) f $1' f 8VLQJ WKH 0$), ILOWHUV JLYHQ LQ f DQG f IRU VSHFWUDO HVWLPDWLRQ UHn TXLUHV DQ LQLWLDO HVWLPDWH RI D^WRf +RZHYHU WKLV WXUQV RXW XQQHFHVVDU\ 3URRI RI f ,QVHUWLQJ f LQWR f \LHOGV SRD0f DA5LR6 Â£ DA5)2A;JA5SRJf ODA5SRJO $f IURP ZKLFK ZH REWDLQ E\ DVVXPLQJ WKDW WKH RfV LQ WKH ULJKWKDQG DQG OHIWKDQG VLGHV DUH LGHQWLFDO LH ERWK DUH RIRPf m)20 L+ O0 5SRJ D05)2D0 DA5SR f J : 5SRJf fÂ§ ODA5SR6O 1H[W ZH UHZULWH WKH )2$ VSHFWUXP VHH ff DV IROORZV De5S JJ+fJ $f m)2$ De5) JJ+fBOD0 ?+ 5 )2 L JA5LRJ f J 5JJA5? ) O J5LJ f 0 5SÂƒ V 5)4J $f D05)2D0 D05n)2D0fV5SRJf a ODA5SRJO ZKLFK LV LGHQWLFDO WR $f +HQFH WKH HTXDOLW\ LQ f LV SURYHG 3URRI RI f 6XEVWLWXWLQJ f LQWR f DQG DIWHU VLPSOH PDQLSXODn WLRQV ZH REWDLQ D0b%D0 a AJ5)%JfD05)%D0f T ODAU5SJJ_ $S%0 -D05)%JfJ5SMD0f a )J5)%JfD05)%D0f A)%0 fÂ§ DA5SJJ $f PAGE 119 %\ ERUURZLQJ WKH QRWDWLRQV GHILQHG LQ ffÂ§f WKH VROXWLRQ mIEP WR $f LV VWUDLJKWIRUZDUG WR REWDLQ f)%0 >LNLLU OQ n _+ n Â0,0 M,0,00 LLAKA LAIAL X II D + LQU N A LKL Lr K ? + A $f 1H[W ZH HYDOXDWH WKH )%$ HVWLPDWRU DV JLYHQ E\ f :H ILUVW FRPSXWH WKH ( L L $ ,0I Zrr\ LPL $f ZKHUH $ ,0W ,.,_ __L__ +A+ SI $f 6XEVWLWXWLQJ $f LQWR f DQG SHUIRUPLQJ VRPH VLPSOH PDQLSXODWLRQV ZH REWDLQ m)%$ >rA __L__ ^AAf LILfLILfr@ >,NLOO OOAOO OOAVOO OODOOL ON__ __L]L__ __L__ __LL__ +ALOO ?YW? __O__ __I ,_ _OI O_ OI OfrLI ,fOALfr LI rfYI 9fr^YbYf ??Y?? ,OI _LI ,_@ $f &RPSDULQJ $f DQG $f FRQILUPV WKDW )%$ DQG )%0 FRLQFLGH +HQFH f IROORZV Â‘ PAGE 120 $33(1',; % 3522) 2) 7+(25(0 :H JLYH EHORZ D SURRI RI 7KHRUHP RQO\ IRU WKH )% FDVH 7KH SURRI IRU WKH )2 FDVH IROORZV D VLPLODU DQG VLPSOHUf SDWWHUQf )URP f DQG f ZH KDYH J DD0 J ttP ZKHUH DQG 6 UHVSHFWLYHO\ DUH GHILQHG DV 6 A L [VfHBLn A )LUVW ZH FDOFXODWH WKH ILUVW DQG VHFRQGRUGHU PRPHQWV RI DQG Â• (^` (^a` e>I ` H^K` R IF R /L - ; Â _Â_f5fHA L /Of N /L Â ; Â _L_f5LfH!fr IH/Of %Of %f %f %f %f %f %f %f %f % f PAGE 121 ZKHUH ^5H]f` LV WKH FRYDULDQFH VHTXHQFH RI HZf RU HZf 1RWH WKDW %f DQG %f DUH GXH WR WKH FLUFXODUO\ V\PPHWULF GLVWULEXWLRQ DVVXPSWLRQ ,W WKHQ IROORZV IURP %f %f DQG &RQGLWLRQ & WKDW 22 OLP /H?V6+? OLP /H^a6a6+? 9 5H^LfHMXML I!H>XLfD0DA %OOf /fÂ§WRR / /fÂ§W RR O ]n LfÂ§fÂ§RR ZKHUH WKH ODVW HTXDOLW\ IROORZV IURP WKH VWDQGDUG UHVXOWV RQ WKH WUDQVIHU RI VSHFWUDO GHQVLWLHV WKURXJK OLQHDU V\VWHPV $PRQJ RWKHUV WKH SUHYLRXV FDOFXODWLRQV LPSO\ WKDW DV / fÂ§} RRJ DQG J WHQG WR DDA DQG R2OP LQ WKH PHDQ VTXDUH VHQVHf UHVSHFWLYHO\ DQG WKHUHIRUH **IL JRHV WR ?RL?D/0t0 +HQFH K%%& DQG K%%$ KDYH WKH VDPH OLPLW DV / fÂ§} RR /HW K GHQRWH D JHQHULF ),5 YHFWRU DQG OHW GHQRWH WKH GHWHUPLQLVWLF YHFWRU WKDW LV WKH OLPLW RI WKH SRVVLEO\ UDQGRPf K ZKHQ / JRHV WR LQILQLW\ 2EVHUYH WKDW IRU DOO PHWKRGV XQGHU VWXG\ WKH DVVRFLDWHG K DQG KRR YHFWRUV VDWLVI\ K+D0 DQG KAD0 %f %\ XVLQJ WKLV REVHUYDWLRQ ZLWK f DQG %Of ZH REWDLQ Â£ K+J D K+6 %f 6LQFH WHQGV WR ]HUR DV / JRHV WR LQILQLW\ LW IROORZV IURP %f WKDW WKH HVWLPDWLRQ HUURU FDQ EH DV\PSWRWLFDOO\ ZULWWHQ DV WR D ILUVWRUGHU DSSUR[LPDWLRQf D fÂ§ D a KAG %f 7KHQ LW UHDGLO\ IROORZV WKDW /( ^D fÂ§ Df` fÂ§ DV / fÂ§} RR %f DQG OLP /( ^?D R_` KA OLP /( ^f eHFMf _KAD0_ HZf %f /r RR /fÂ§!RR / DQG WKH SURRI LV FRQFOXGHG PAGE 122 $33(1',; & 3522) 2) 7+(25(0 3URRI RI f :H ILUVW FRQVLGHU WKH ELDV IRU WKH )%& %\ XVLQJ f DQG %f ZH REWDLQ 3 B D$IU/)%r UQ L? m)%& a D 77:7L n &f 5S%D$L ,Q ZKDW IROORZV ZH XVH WKH V\PERO a WR GHQRWH DQ fDV\PSWRWLF HTXDOLW\f WKDW KROGV WR ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQ $ VWUDLJKWIRUZDUG PDQLSXODWLRQ RI &Of \LHOGV m)%& B D D05I% A A D0A )% !+Ga D0A)%D0 D0A)%D0 DA5A55)%f5A D05)%D0 ; 7DA5A DA5SÂD0 DA5D0 D5D0B DA5+5)%5fAA DA5[DZ U DA5[D0 A5A5IE 5f5n Dt5r DA5A5 5AfD0 ,7 L DA5)%D0fD05 t$Lf DA5[D P De5A DA5a[D 0 DA5fDA5a5)% a 5f5aOD0 DA5[D0f &f ZKLFK LQ WXUQ LPSOLHV WKDW ( ^m)%& m` ( %& m DA55)%5aÂfDA5D0f DA55)%5aD0fDA5Âf DA5[D0f &f PAGE 123 ,OO 1H[W ZH QRWH WKDW /O 5)% >^DD0HXO HOff DD0H"XO HOff+ Q DD0HMf HOff Dt0HMXO HOff+ ?D?D0t0 mrÂf 0RÂ+ Â£+f Â(>Jf:Hnf 0cf@ A &f :H DOVR UHPLQG WKH UHDGHU RI %f DQG WKH DVVXPSWLRQ WKDW HXQf DQG HZQf KDYH ]HUR WKLUGRUGHU PRPHQWV 6LQFH HfOf -8/Of &f ZKLFK LV OLNH f ZH KDYH m A/ O A A Ha0/@N H0/ %\ XVLQJ WKHVH IDFWV DORQJ ZLWK &f ZH FDQ ZULWH &f D05 D0f ( ^Â£S%& a m` eMD 5nD0f DMA5 D0fD6+ Dff5 r De5f DA5 DrG FLrGfDA5 [D0f _DA5 D0f4 Â£+f5 OP ?( ^DrDA5fDA5D0f Â£rDA5Â£fDA5fÂ£fDA5fD0f mDm5 [DZf DA5A Â£DeI5 D0fDA5 fÂ£f5 [D0f L PAGE 124 DrD5D0fD5LDf Â£rDA5D0fD5fD5Af DDA5BD0f5f Â£DA5fD0fÂ:5ff_ OD^D+05 rPf( ^_Dr 5fA DA5D0f5af` &f %\ WKH &DXFK\6FKZDUW] LQHTXDOLW\ WKH TXDQWLW\ EHWZHHQ WKH FXUO\ SDUHQWKHVHV LQ &f LV QHJDWLYH DQG VR LV LWV H[SHFWDWLRQ 7KH ELDV IRU WKH )2& VSHFWUDO HVWLPDWH FDQ EH REWDLQHG E\ UHSODFLQJ WKH )% VDPSOH FRYDULDQFH 5SS LQ &f&f E\ WKH )2 VDPSOH FRYDULDQFH PDWUL[ 5SR! DV GHILQHG LQ f DQG E\ IROORZLQJ D VLPLODU WUHDWPHQW ZH GLG LQ &f 7KH UHVXOW LV DV IROORZV WR D VHFRQGRUGHU DSSUR[LPDWLRQf DA5D0fe ^m)2& D` a ^ODA5AI Dt59.A5rf` &f +HQFH WR ZLWKLQ D VHFRQGRUGHU DSSUR[LPDWLRQ WKH ELDV RI WKH )%& LV RQH KDOI WKDW RI WKH )2& DQG f IROORZV Â‘ 3URRI RI f $JDLQ ZH FRQVLGHU )%$ ILUVW 2EVHUYH WKDW 4IE$ fÂ§ 4 9=f 7KHQ E\ f DQG %f ZH KDYH VHH &ff m)%$ m VP4IEDA D04)%$D0 DA44)%$4f4A DA40 DP4 DA4 -D0 DIF4A. &7+4IED 4f4aOD0 DA4nA$Lf &f 7KHUHIRUH ZH REWDLQ DZ4 rP"( ^$)ED D` ( ^DA4B4)%$4BfDA4BD0f DA4f4)%$4AD0fDA4ff_ &f PAGE 125 1H[W QRWH WKDW 4IED 5IE ** 5IE LW >JJ+ JJ@ _D_D0DA ADr6DA _Â£rDA _DD06 ADD0 _0DI /fÂ§ / 9 & 0=fHZ ZA@ +D0DA ADD0Â£ UDD+ B a6+ Br b DD0m DGD0 Â ( 0fH": c:ce@ f rr P / K $L &OOf ZKHUH ZH KDYH PDGH XVH RI &f DQG %f%f 8VLQJ DJDLQ WKH DVVXPSWLRQ RI ]HUR WKLUGRUGHU PRPHQWV RI HZQf DQG HXQf DQG FRPELQLQJ &f DQG &OOf \LHOGV WR D VHFRQGRUGHU DSSUR[LPDWLRQf ( ^mIE$ D` a &f /LNHZLVH E\ UHSODFLQJ 4IED E\ 4I2$ LQ &f&f IROORZHG E\ VRPH VWUDLJKWIRUn ZDUG PDQLSXODWLRQV ZH KDYH WR D VHFRQGRUGHU DSSUR[LPDWLRQf fA^f)2$ a &f DQG WKH SURRI RI f LV FRPSOHWH 7R PRWLYDWH WKH QRUPDOL]LQJ IDFWRU / XVHG LQ ERWK f DQG f ZH PHQn WLRQ WKH IDFW WKDW ERWK DQG 6 DUH O9/f DQG WKLV LPSOLHV WKDW WKH VHFRQGRUGHU DSSUR[LPDWLRQ SUHYLRXVO\ XVHG LV /f Â‘ PAGE 126 $33(1',; 3522) 2) 7+(25(0 /HPPD 0fÂ§O0 00 0O0M 'Of 3URRI 0: 0fÂ§O 0fÂ§O IN!N Mf N N M M IF 0fÂ§O 0fÂ§O 0fÂ§O MfÂ§ 'f MfÂ§ NfÂ§M M IF /HW V NfÂ§ M 1RWH WKDW WKH ILUVW WHUP RI WKH ULJKW VLGH RI 'f FRUUHVSRQGV WR V DQG WKH VHFRQG WHUP FRUUHVSRQGV WR V 6LQFH N 0 fÂ§ DQG M 0 fÂ§ LW LV REYLRXV WKDW 0 fÂ§ N V M V IRU V N V M V 0 fÂ§ 0 fÂ§ _V_ fÂ§ IRU V 7KHUHIRUH ZH FDQ UHZULWH 'f DV 0fÂ§O 0fÂ§O 0?V?O nc!-I^NVf e Pf V N V V fÂ§0fÂ§f N 0fÂ§O 0fÂ§O 0fÂ§O 0fÂ§VfÂ§O AAINAVf INfaVf! V N V V O IF ZKLFK FRQFOXGHV WKH SURRI RI /HPPD 'f 'f PAGE 127 3URRI RI 7KHRUHP %\ PDNLQJ XVH RI /HPPD ZH REWDLQ 00 DIIZfUDZf DOUADL N 00 ((AK N Â 0 0 0fÂ§ 0V ((UmffHn ( ( U0fHf V N V V O IF 0 ( V ^0Of ZLWK 8V GHILQHG LQ f ,I U WKHQ ZH KDYH 0fÂ§VfÂ§ +6 Ur!IFr! IRU V 'f N 7DNLQJ FRPSOH[ FRQMXJDWLRQ RI ERWK VLGHV \LHOGV 0fÂ§VfÂ§ 0fÂ§VfÂ§ }ff ( U3} ( Ufrr IF IF 0fÂ§O Ur!rV 'f O V DQG WKH SURRI LV FRPSOHWH PAGE 128 5()(5(1&(6 >@ 7 : $QGHUVRQ $V\PSWRWLFDOO\ HIILFLHQW HVWLPDWLRQ RI FRYDULDQFH PDWULFHV ZLWK OLQHDU VWUXFWXUH 7KH $QQDOV RI 6WDWLVWLFV f fÂ§ >@ 6 3 $SSOHEDXP DQG &KDSPDQ $GDSWLYH DUUD\V ZLWK PDLQ EHDP FRQn VWUDLQWV ,((( 7UDQVDFWLRQV RQ $QWHQQDV DQG 3URSDJDWLRQ 6HSWHPn EHU >@ $ $XVKHUPDQ $ .R]PD / :DONHU + 0 -RQHV DQG ( & 3RJJLR 'HYHORSPHQWV LQ UDGDU LPDJLQJ ,((( 7UDQVDFWLRQV RQ $HURVSDFH DQG (OHFWURQLF 6\VWHPV f -XO\ >@ 5 %HQLW] 3UHOLPLQDU\ UHVXOWV LQ DGDSWLYH KLJK GHILQLWLRQ LPDJLQJ IRU VWDn WLRQDU\ WDUJHWV 0,7 /LQFROQ /DERUDWRU\ 3URMHFW 5HSRUW $67 1RYHPEHU >@ 5 % %ODFNPDQ DQG : 7XNH\ 7KH 0HDVXUHPHQW RI 3RZHU 6SHFWUD IURP WKH 3RLQW RI 9LHZ RI &RPPXQLFDWLRQV (QJLQHHULQJ 'RYHU 1HZ PAGE 129 >@ $ &DG]RZ 6LJQDO HQKDQFHPHQW D FRPSRVLWH SURSHUW\ PDSSLQJ DOJRULWKP ,((( 7UDQVDFWLRQV RQ $FRXVWLFV 6SHHFK DQG 6LJQDO 3URFHVVLQJ ffÂ§ -DQXDU\ >@ &DSRQ +LJK UHVROXWLRQ IUHTXHQF\ZDYHQXPEHU VSHFWUXP DQDO\VLV 3URFHHGLQJV RI WKH ,((( f fÂ§ $XJXVW >@ &DSRQ &RUUHFWLRQ WR fSUREDELOLW\ GLVWULEXWLRQV IRU HVWLPDWRUV RI WKH IUHTXHQF\ ZDYHQXPEHU VSHFWUXPf 3URFHHGLQJV RI WKH ,((( f -DQXDU\ >@ &DSRQ 0D[LPXPOLNHOLKRRG VSHFWUDO HVWLPDWLRQ LQ 6 +D\NLQ HGLWRU 1RQn OLQHDU 0HWKRGV RI 6SHFWUDO $QDO\VLV 1HZ PAGE 130 >@ / *ULIILWKV DQG & : -LP $Q DOWHUQDWLYH DSSURDFK WR OLQHDUO\ FRQVWUDLQHG DGDSWLYH EHDPIRUPLQJ ,((( 7UDQVDFWLRQV RQ $QWHQQDV DQG 3URSDJDWLRQ $3 -DQXDU\ >@ *XSWD +LJKUHVROXWLRQ UDGDU LPDJLQJ XVLQJ OLQHDU SUHGLFWLRQ ,((( 7UDQVDFWLRQV RQ $QWHQQDV DQG 3URSDJDWLRQ Of -DQXDU\ >@ ( +DQQDQ DQG % :DKOEHUJ &RQYHUJHQFH UDWHV IRU LQYHUVH 7RHSOLW] PDWUL[ IRUPV -RXUQDO RI 0XOWLYDULDWH $QDO\VLV 2FWREHU >@ 6 +D\NLQ HGLWRU $UUD\ 6LJQDO 3URFHVVLQJ 3UHQWLFH+DOO (QJOHZRRG &OLIIV 1>@ 0 +RQLJ DQG 8 0DGKRZ %OLQG DGDSWLYH PXOWLXVHU GHWHFWLRQ ,((( 7UDQVDFn WLRQV RQ ,QIRUPDWLRQ 7KHRU\ f -XO\ >@ 5 $ +RUQ DQG & 5 -RKQVRQ 0DWUL[ $QDO\VLV &DPEULGJH 8QLYHUVLW\ 3UHVV &DPEULGJH >@ < +XD +LJK UHVROXWLRQ LPDJLQJ RI FRQWLQXRXVO\ PRYLQJ REMHFW XVLQJ VWHSSHG IUHTXHQF\ UDGDU 6LJQDO 3URFHVVLQJ Of -DQXDU\ >@ $ -DNREVVRQ 7 (NPDQ DQG 3 6WRLFD &DSRQ DQG $3(6 VSHFWUXP HVWLPDn WLRQ IRU UHDOYDOXHG VLJQDOV 3URFHHGLQJV RI WKH WK ,((( '63 :RUNVKRS %U\FH &DQ\RQ $XJXVW >@ 3 9 .DEDLOD 2Q RXWSXWHUURU PHWKRGV IRU V\VWHP LGHQWLILFDWLRQ ,((( 7UDQVn DFWLRQV RQ $XWRPDWLF &RQWURO Of -DQXDU\ >@ 6 0 .D\ 0RGHUQ 6SHFWUDO (VWLPDWLRQ 7KHRU\ DQG $SSOLFDWLRQ 3UHQWLFH+DOO (QJOHZRRG &OLIIV 1>@ 6 0 .D\ DQG 6 / 0DUSOH -U 6SHFWUXP DQDO\VLV D PRGHUQ SHUVSHFWLYH 3URFHHGLQJV RI WKH ,((( f fÂ§ 1RYHPEHU >@ 6 % .HVOHU HGLWRU 0RGHUQ 6SHFWUXP $QDO\VLV ,, ,((( 3UHVV 1HZ PAGE 131 >@ + /L 3 6WRLFD DQG /L &RPSXWDWLRQDOO\ HIILFLHQW PD[LPXP OLNHOLKRRG HVWLn PDWLRQ RI VWUXFWXUHG FRYDULDQFH PDWULFHV 3URFHHGLQJV RI WKH ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ $FRXVWLFV 6SHHFK DQG 6LJQDO 3URFHVVLQJ SDJHV 6HDWn WOH 0D\ >@ + /L 3 6WRLFD DQG /L &RPSXWDWLRQDOO\ HIILFLHQW PD[LPXP OLNHOLKRRG HVWLPDn WLRQ RI VWUXFWXUHG FRYDULDQFH PDWULFHV ,((( 7UDQVDFWLRQV RQ 6LJQDO 3URFHVVLQJ f 0D\ >@ + /L 3 6WRLFD /L DQG $ -DNREVVRQ 2Q WKH SHUIRUPDQFH DQDO\VLV RI IRUZDUG RQO\ DQG IRUZDUGEDFNZDUG PDWFKHGILOWHUEDQN VSHFWUDO HVWLPDWRUV 3URFHHGLQJV RI VW $QQXDO $VLORPDU &RQIHUHQFH RQ 6LJQDOV 6\VWHPV DQG &RPSXWHUV SDJHV 3DFLILF *URYH &DOLIRUQLD 1RYHPEHU >@ /L DQG 3 6WRLFD $Q DGDSWLYH ILOWHULQJ DSSURDFK WR VSHFWUDO HVWLPDWLRQ DQG 6$5 LPDJLQJ ,((( 7UDQVDFWLRQV RQ 6LJQDO 3URFHVVLQJ f -XQH >@ =6 /LX + /L DQG /L (IILFLHQW LPSOHPHQWDWLRQ RI &DSRQ DQG $3(6 IRU VSHFWUDO HVWLPDWLRQ ,((( 7UDQVDFWLRQV RQ $HURVSDFH DQG (OHFWURQLF 6\VWHPV f 2FWREHU >@ / /MXQJ 6\VWHP ,GHQWLILFDWLRQ 7KHRU\ IRU WKH 8VHU 3UHQWLFH+DOO (QJOHZRRG &OLIIV 1>@ 6 / 0DUSOH -U 'LJLWDO 6SHFWUDO $QDO\VLV ZLWK $SSOLFDWLRQV 3UHQWLFH+DOO (QJOHZRRG &OLIIV 1>@ & 7 0XOOLV DQG / / 6FKDUI 4XDGUDWLF HVWLPDWRUV RI WKH SRZHU VSHFWUXP LQ 6 +D\NLQ HGLWRU $GYDQFHV LQ 6SHFWUXP $QDO\VLV DQG $UUD\ 3URFHVVLQJ (QJOHZRRG &OLIIV 13UHQWLFH+DOO >@ & 0XQVRQ -U 2f%ULDQ DQG : -HQNLQV $ WRPRJUDSKLF IRUPXODWLRQ RI VSRWOLJKW PRGH V\QWKHWLF DSHUWXUH UDGDU 3URFHHGLQJV RI WKH ,((( f $XJXVW >@ % 0XVLFXV )DVW 0/0 SRZHU VSHFWUXP HVWLPDWLRQ IURP XQLIRUPO\ VSDFHG FRUn UHODWLRQV ,((( 7UDQVDFWLRQV RQ $FRXVWLFV 6SHHFK 6LJQDO 3URFHVVLQJ 2FWREHU >@ 5 2QQ DQG $ 2 6WHLQKDUGW 0XOWLZLQGRZV VSHFWUXP HVWLPDWLRQ D OLQHDU DOJHEUDLF DSSURDFK ,QWHUQDWLRQDO -RXUQDO RQ $GDSWLYH &RQWURO DQG 6LJQDO 3URn FHVVLQJ >@ % 3RUDW 6RPH DV\PSWRWLF SURSHUWLHV RI WKH VDPSOH FRYDULDQFH RI *DXVVLDQ DXWRUHJUHVVLYH PRYLQJDYHUDJH SURFHVVHV -RXUQDO RI 7LPH 6HULHV $QDO\VLV f >@ 5LHGHO DQG $ 6LGRUHQNR 0LQLPXP ELDV PXOWLSOH WDSHU VSHFWUDO HVWLPDWLRQ ,((( 7UDQVDFWLRQV RQ 6LJQDO 3URFHVVLQJ f fÂ§ -DQXDU\ >@ ( $ 5RELQVRQ $ KLVWRULFDO SHUVSHFWLYH RI VSHFWUXP HVWLPDWLRQ 3URFHHGLQJV RI WKH ,((( f 6HSWHPEHU >@ 5 5R\ DQG 7KRPDV .DLODWK (635,7HVWLPDWLRQ RI VLJQDO SDUDPHWHUV YLD URn WDWLRQDO LQYDULDQFH WHFKQLTXHV ,((( 7UDQVDFWLRQV RQ $FRXVWLFV 6SHHFK DQG 6LJQDO 3URFHVVLQJ f -XO\ PAGE 132 >@ 5 2 6FKPLGW 0XOWLSOH HPLWWHU ORFDWLRQ DQG VLJQDO SDUDPHWHU HVWLPDWLRQ ,((( 7UDQVDFWLRQV RQ $QWHQQDV DQG 3URSDJDWLRQ ffÂ§ 0DUFK >@ % 6FKRGRUI DQG % :LOOLDPV $UUD\ SURFHVVLQJ WHFKQLTXHV IRU PXOWLXVHU GHWHFWLRQ ,((( 7UDQVDFWLRQV RQ &RPPXQLFDWLRQV f fÂ§ 1RYHPEHU >@ % 6FKRGRUI DQG % :LOOLDPV $ FRQVWUDLQHG RSWLPL]DWLRQ DSSURDFK WR PXOWLXVHU GHWHFWLRQ ,((( 7UDQVDFWLRQV RQ 6LJQDO 3URFHVVLQJ f fÂ§ -DQXDU\ >@ $ 6FKXVWHU 7KH SHULRGRJUDP DQG LWV RSWLFDO DQDORJ\ 3URFHHGLQJV RI WKH 5R\DO 6RFLHW\ RI /RQGRQ 6HULHV $7 -XQH >@ 7 6RGHUVWURP DQG 3 6WRLFD 6\VWHP ,GHQWLILFDWLRQ 3UHQWLFH+DOO /RQGRQ >@ 3 6WRLFD % )ULHGODQGHU DQG 7 6RGHUVWURP $SSUR[LPDWH PD[LPXPOLNHOLKRRG DSSURDFK WR $50$ VSHFWUDO HVWLPDWLRQ ,QWHUQDWLRQDO -RXUQDO RI &RQWURO f fÂ§ >@ 3 6WRLFD $ -DNREVVRQ DQG /L 0DWFKHGILOWHU EDQN LQWHUSUHWDWLRQ RI VRPH VSHFWUDO HVWLPDWRUV 6LJQDO 3URFHVVLQJ ffÂ§ $SULO >@ 3 6WRLFD + /L DQG /L $PSOLWXGH HVWLPDWLRQ RI VLQXVRLGDO VLJQDOV 6XUYH\ QHZ UHVXOWV DQG DQ DSSOLFDWLRQ 7R DSSHDU LQ ,((( 7UDQVDFWLRQV RQ 6LJQDO 3URFHVVLQJ >@ 3 6WRLFD + /L DQG /L $ QHZ GHULYDWLRQ RI WKH $3(6 ILOWHU 7R DSSHDU LQ ,((( 6LJQDO 3URFHVVLQJ /HWWHUV >@ 3 6WRLFD + /L DQG /L 6SHFWUDO HVWLPDWLRQ DQG UDGDU LPDJLQJ 3URFHHGLQJV RI WK ,6&,( ,QWHUQDWLRQDO 6\PSRVLXP RQ 6WRFKDVWLF 6\VWHPV 7KHRU\ DQG ,WV $SSOLFDWLRQV .\RWR -DSDQ 1RYHPEHU >@ 3 6WRLFD + /L DQG /L $PSOLWXGH HVWLPDWLRQ ZLWK DSSOLFDWLRQV WR V\VWHP LGHQWLILFDWLRQ 3URFHHGLQJV RI WKH ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ $FRXVWLFV 6SHHFK DQG 6LJQDO 3URFHVVLQJ SDJHV 3KRHQL[ $UL]RQD 0DUFK >@ 3 6WRLFD DQG 5 / 0RVHV ,QWURGXFWLRQ WR 6SHFWUDO $QDO\VLV 3UHQWLFH+DOO 8SSHU 6DGGOH 5LYHU 1>@ 3 6WRLFD DQG 7 6RGHUVWURP 2Q UHSDUDPHWHUL]DWLRQ RI ORVV IXQFWLRQV XVHG LQ HVWLPDWLRQ DQG WKH LQYDULDQFH SULQFLSOH 6LJQDO 3URFHVVLQJ $XJXVW >@ 7KRPVRQ 6SHFWUXP HVWLPDWLRQ DQG KDUPRQLF DQDO\VLV 3URFHHGLQJV RI WKH ,((( f 6HSWHPEHU >@ 7 8OU\FK DQG 7 1 %LVKRS 0D[LPXP HQWURS\ VSHFWUDO DQDO\VLV DQG DXWRUHn JUHVVLYH GHFRPSRVLWLRQ 5HYLHZV RI *HRSK\VLFV DQG 6SDFH 3K\VLFV )HEUXDU\ >@ % 9DQ 9HHQ DQG 0 %XFNOH\ %HDPIRUPLQJ $ YHUVDWLOH DSSURDFK WR VSDWLDO ILOWHULQJ ,((( 6LJQDO 3URFHVVLQJ 0DJD]LQH SDJHV $SULO PAGE 133 >@ + :DQJ DQG / &DL 2Q DGDSWLYH VSDWLDOWHPSRUDO SURFHVVLQJ IRU DLUERUQH VXUYHLOODQFH UDGDU V\VWHPV ,((( 7UDQVDFWLRQV RQ $HURVSDFH DQG (OHFWURQLF 6\VWHPV f -XO\ >@ + :DQJ DQG + :DNHILHOG 6LJQDOVXEVSDFH DSSUR[LPDWLRQ IRU OLQH VSHFWUXP HVWLPDWLRQ 3URFHHGLQJV RI WKH ,QWHUQDWLRQDO &RQIHUHQFH RQ $FRXVWLFV 6SHHFK DQG 6LJQDO 3URFHVVLQJ SDJHV 'DOODV 7; $SULO >@ :DUG 6SDFHWLPH DGDSWLYH SURFHVVLQJ IRU DLUERUQH UDGDU 7HFKQLFDO 5HSRUW /LQFROQ /DERUDWRU\ 0,7 'HFHPEHU >@ % :LGURZ 3 ( 0DQWH\ / *ULIILWKV DQG % % *RRGH $GDSWLYH DQWHQQD V\VWHPV 3URFHHGLQJV RI WKH ,((( f fÂ§ 'HFHPEHU >@ 0 :LONHV DQG 0 + +D\HV ,WHUDWHG 7RHSOLW] DSSUR[LPDWLRQ RI FRYDULDQFH PDWULFHV 3URFHHGLQJV RI WKH ,QWHUQDWLRQDO &RQIHUHQFH RQ $FRXVWLFV 6SHHFK DQG 6LJQDO 3URFHVVLQJ SDJHV 1HZ PAGE 134 %,2*5$3+,&$/ 6.(7&+ +RQJELQ /L ZDV ERUQ RQ 0DUFK +H UHFHLYHG WKH %6 DQG 06 GHJUHHV LQ HOHFWULFDO HQJLQHHULQJ IURP WKH 8QLYHUVLW\ RI (OHFWURQLF 6FLHQFH DQG 7HFKQRORJ\ RI &KLQD 86(7&f LQ -XO\ DQG $SULO UHVSHFWLYHO\ +H ZDV DIILOLDWHG DV DQ $VVLVWDQW /HFWXUHU DW WKH 8(67& IURP WR 6LQFH KH KDV EHHQ ZRUNLQJ WRZDUG KLV 3K' GHJUHH LQ HOHFWULFDO HQJLQHHULQJ DW WKH 8QLYHUVLW\ RI )ORULGD PAGE 135 , FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ -LDQ /L &KDLUPDQ $VVRFLDWH 3URIHVVRU RI (OHFWULFDO DQG &RPSXWHU (QJLQHHULQJ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ -RVH & 3ULQFLSH 3URIHVVRU RI (OHFWULFDO DQG &RPSXWHU (QJLQHHULQJ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ :LOOLDP : (GPRQVRQ $VVLVWDQW 3URIHVVRU RI (OHFWULFDO DQG &RPSXWHU (QJLQHHULQJ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ )UHGHULFN 7D\ORU 3URIHVVRU RI (OHFWULFDO DQG &RPSXWHU (QJLQHHULQJ FHUWLI\ WKDW KDYH UHDG WKLV VWXG\ DQG WKDW LQ P\ RSLQLRQ LW FRQIRUPV WR DFFHSWDEOH VWDQGDUGV RI VFKRODUO\ SUHVHQWDWLRQ DQG LV IXOO\ DGHTXDWH LQ VFRSH DQG TXDOLW\ DV D GLVVHUWDWLRQ IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 'DYLG & :LOVRQ 3URIHVVRU RI 0DWKHPDWLFV PAGE 136 7KLV GLVVHUWDWLRQ ZDV VXEPLWWHG WR WKH *UDGXDWH )DFXOW\ RI WKH &ROOHJH RI (QJLQHHULQJ DQG WR WKH *UDGXDWH 6FKRRO DQG ZDV DFFHSWHG DV SDUWLDO IXOILOOPHQW RI WKH UHTXLUHPHQWV IRU WKH GHJUHH RI 'RFWRU RI 3KLORVRSK\ 0D\ :LQIUHG 0 3KLOOLSV 'HDQ &ROOHJH RI (QJLQHHULQJ 0LKUDQ 2KDQLDQ 'HDQ *UDGXDWH 6FKRRO 46 Figure 4.1: SAR images obtained from the ERIM data by using (a) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES. 2-D FFT, (b) 93 (c) (d) Figure 6.2: Pole-zero diagrams for ARMA test cases, (a) ARMA1; (b) ARMA2; (c) ARM A3; (d) ARMA4. In the following numerical examples, we set N 256 (shorter data lengths have also been considered, and the results are similar) and M 50 (see Sections 3.5.1 and 5.5.2 for how to choose M). The performances of the standard and the Capon methods for each of the four ARMA models are shown in Figures 6.5(a) to 6.5(d), where the curves show the mean-squared errors (MSE) of the covariance estimates, normalized with respect to r(0), versus the time-lag of the covariance sequences. (Only the results from the exact Capon method are demonstrated owning to space limitation.) The MSE values are based on 100 independent realizations. It has been found that the Capon method generally gives better results than the standard method, especially for large time-lags. We also note that, when the poles are close to the unit circle, as in Figure 6.5(b), the performance differences between the two methods are not so large as in the other cases. (Though, the Capon method is still better than the standard method.) ACKNOWLEDGEMENTS I would like to express my sincere gratitude to my advisor, Dr. Jian Li, for her constant support, encouragement, enthusiasm, and patience in guiding this research. My deepest appreciation goes to Dr. Petre Stoica for his numerous insightful remarks and suggestions which comprehensively influenced this work. Special thanks are due to Drs. Jose C. Principe, Fred J. Taylor, William W. Edmonson, and David C. Wilson for serving on my supervisory committee and for their contribution to my graduate education at the University of Florida. I also wish to thank Zhaoqiang Bi, Robert Stanfill, and other fellow graduate students with whom I had the great pleasure of interacting. Drs. Zheng-She Liu and Guoqing Liu have my gratitude for sharing many interesting discussions with me. I would like to gratefully acknowledge all the people who helped me during my Ph.D. program. iii 86 ARMA(4,2)(N=256, M=51, 100 realizations.) ARMA(4,2)(N=32, M=11,100 realizations.) (a) (b) Figure 6.1: Power spectral density estimates by using Capon-1 and Capon-2. The plots are the averages of 100 independent realizations, (a) N = 256, M = 50; (b) iV = 32,M = 10. signal, while the solid and the dashdotted lines, respectively, indicate the Capon-1 and Capon-2 estimates. It is obvious that Capon-2 is highly biased and suffers from a significant power loss. This observation has also been made with other signals, especially if the signal is a narrowband signal. An explanation of this behavior follows. The calculation of the filter bandwidth in Capon-2 is applicable only if the Capon filter is a narrowband filter. Recall that the Capon method aims to find the Capon filter that minimizes the total output power of the overall frequency band while passes the current frequency u> undistorted. No effort has been taken to make sure that the Capon filter is narrowband. Let the lobe of the Capon filter frequency response where the current frequency of interest uj is located be called as the mainlobe, while all the others are called the sidelobes. It has been found that the steering frequency uj is not necessarily at the maximum or the center of the mainlobe [75] [52], Furthermore, if the input signal is a narrowband signal, there may exist sidelobes, located at frequency bands where the power level of the input signal is low, that are even larger than the mainlobe of the Capon filter frequency response. Note 24 3.3 Analyses of MAFI Approaches 3.3.1 Computational Complexity Let RpQ2 and Rpg 2 denote the Hermitian square roots of the positive definite matrices RpQ and Rpg, respectively. Define //.i(iu) = RpQ awf(o;), (3.44) M2(w) = Rpo (3.45) Then FOC and FOA can be expressed as relatively simple functions of /^(w) and /z2(u;) (see (3.29) and (3.35)): FOcM= IIalHII2 and foaM = Ml hm2m IImiMII2 (IImiHII2 IIm2M||2 Im?(w)m2MI2) Applying the matrix inversion lemma to (3.32) yields i -i QFBAM ^FB %BG(w) G (w)RpBG(kO ^ Gff((j)Rpg, (3.46) (3.47) (3.48) where I is the 2x2 identity matrix. Next define 'lM = Rpg/2aM(w), v2(u) = RpjfgH, u3(u) = Rpg/2g(w). Then the FBC and FBA spectral estimators can be expressed as (see (3.29), and (3.48)): fbcM Ui(u})u2{uj) IkiHII2 (3.49) (3.50) (3.51) (3.35), (3.52) 22 where Q(w) denotes either Qfqa or QfbA which corresponds to FOA or FBA. Consequently, the APES estimate of a(u) is given by (see (3.22) and (3.26)) aM(w)Q-1(w)g(w) apesM = (3.35) 3.2.3 Another Matched Filter Equations (3.4) and (3.7) suggest another way to estimate the FO and FB esti mate of Q(w) (in what follows we sometimes omit the dependence on u> for notational convenience): QfomM \l~i = jYl [^(0 (^)aM(w)eJ,i] [y(l) (u)aM(w)eJji]H =o Rpo *ga^ -aMgH + ||2aMa^, H ~|2. (3.36) a FO QfbmM L-1 = [(y(0 (w)aAf(w)e,wi) (y(0 a(u)aM(u)ejul) ^ 1=0 H + (y(0 a(w)aM(w)eJU,j (y(0 ~ (w)aM(w)eJiJ^ H = RpB 1 g + ag H 1 aM 2&M l H A*g + & g + \\aMa%f, (3.37) 2 FB where a(u) and a(u>) a*(o;)e_^Ar_1^ denote some estimates of a(u>) and a(co), respectively. By using the matrix inversion lemma (twice), one can see that the last and the third terms of (3.36) or (3.37) can be dropped without affecting the matched filter vector. Then, by using the matrix inversion lemma once again, we have OpoaM = (RfO *ga^) 1 -1 1 R-fo + O aM 1 dda^RpQg _ _ RpQaM ~ &*{aM Rfo S) Rpo ajw + a* (a^rRpQaAf )RpQg 1 A*a" RpJjg (3.38) 70 Estimator M APES1 N/A < M < N/2 APESK N/A < M < N/3 MAFI1 N/8 < M < 27V/5 Caponl or CaponK N/8 where the input u(n) is a sinusoidal (probing) signal K u(n) = ^2,lke]Ukn, n = 0, k=l and the transfer function is rational: H(-u = B(z ') = biz l + ... + bqz l A{z~l) 1 + aiz-1 + ... + apz~P We assume that K > p + q. (5.72) (5.73) (5.74) Even if p and q were unknown, K could still be chosen sufficiently large to satisfy (5.74). The problem of interest in this section is to estimate {a}^=:1 and {6^}y=1 from 5.6.1 System Identification Using Amplitude Estimation The commonly-used Output Error Method (OEM) does not model v(n) and obtains estimates of {aj}=1 and {bj}9j=l by minimizing the criterion Nl <^OEM(a, b) = Hn) ~ H{z~l)u{n) |2 (5.75) 710 where a = [ fll ... ap ]T and b = [ bx ... bq ]T Let a*(a,b) = jkH(eJUJk). (5.76) 113 Next note that Qfba = R-fb GG// = R-fb it [ggH + gg7/] = |a|2aMa^ + ^a*Sa^ + |*5a^ + |aaM L1 2L + V7 !C MZ)ew(0 + w(0^(0] H2aMa^ ^aaM" 1=0 1 raaH + 1 ~SH 1 _* % // --aaM5 --adaM-- E M)e?W + Â¡W0Â¡Â£(0] **" + m" 1=0 2 L h Ai (C.ll) where we have made use of (C.4) and (B.1)-(B.2). Using again the assumption of zero third-order moments of ew(n) and eu(n), and combining (C.10) and (C.ll) yields (to a second-order approximation) E {fbA a} ~ 0. (C.12) Likewise, by replacing Qfba by QfOA in (C.9)-(C.10) followed by some straightfor ward manipulations, we have (to a second-order approximation) ^{FOA ~ 0, (C.13) and the proof of (3.58) is complete. To motivate the normalizing factor L used in both (3.57) and (3.58), we men tion the fact that both 6 and S are 0(l/VL) and this implies that the second-order approximation previously used is 0(1/L). Power Spectral Density-dB Power Spectral Density-dB 94 ARMA1 ARMA2 (a) ARMA3 (C) (b) ARMA4 (d) Figure 6.3: True power spectral densities, (a) ARMA1; (b) ARMA2; (c) ARMA3; (d) ARMA4. 100 (a) (b) Figure 6.7: 10 superimposed realizations of the MA covariance sequence estimates with N = 64 and M = 6. (a) The standard method; (b) The Capon method. We use both the standard and the Capon methods to determine its covariance esti mates. The results are shown in Figures 6.7(a) and 6.7(b) where N 64 and M = 6, and where 10 superimposed realizations for both methods are displayed. As expected, the Capon covariance estimates decay to almost zero after k = 5, while the standard covariance estimates are much more erratic. Consequently the inference that the process under study is an MA(4) is easier to make by using the Capon covariance estimator. 6.6 Summary The Capon method for covariance sequence estimation makes use of Fourier inversion of the Capon spectral estimates. It is shown that the Capon covariance estimates are usually more accurate than the standard biased sample covariance estimates, especially for large lags. The Capon spectrum is equivalent to an AR spectrum and hence the corresponding covariance sequence can be exactly calculated via the inverse Levinson-Durbin algorithm. Since the spectral factorization needed for the exact Capon covariance estimation is in general computationally expensive, CHAPTER 7 CONCLUSIONS 7.1 Summary Remarks In this work, we have discussed using MAtched-FIlterbank (MAFI) approaches for spectral analysis and parameter estimation. Specifically, the problems of complex spectral estimation, sinusoidal amplitude estimation, and covariance sequence esti mation have been addressed. In Chapter 3, we first introduced the MAFI concept, which is of primary importance to this study, by choosing the filterbank as matched-filterbank, i.e., each filter in the filterbank is a matched filter. Since we do not model the observation noise exactly, there are various ways of estimating the covariance matrix of the observation noise, which lead to different MAFI estimators for specific estimation purposes. For complex spectral estimation, it was shown in Chapter 3 that the Capon and APES estimators, while originally not derived in the MAFI framework, are both members of the MAFI complex spectral estimators. Moreover, the MAFI concept may be used to devise other interesting spectral estimators. Although Capon and APES are both asymptotically efficient and are of similar computational complexity, their performances are quite different for finite length of data samples. Specifically, we proved that, to within a second-order approximation, Capon underestimates the true spectrum whereas APES is unbiased, and that the bias of the forward-backward Capon is one half that of the forward-only Capon. These results are believed to pro vide compelling reasons for preferring APES to Capon in most practical applications. It should be stressed that the better estimation accuracy obtained by Capon or APES over the Fourier-based methods is achieved at the cost of computational 102 APPENDIX C PROOF OF THEOREM 3.3.2 Proof of (3.57): We first consider the bias for the FBC. By using (3.28) and (B.13), we obtain P _1 aAfrLFB rn i\ FBC ~ a = -TTWTi (C-1) RpBaAi In what follows, we use the symbol ~ to denote an asymptotic equality that holds to within a second-order approximation. A straightforward manipulation of (C.l) yields FBC a aM(RfB ^ ^ aM^ FB >H-d~ 1 + aM^FBaM aM-^FBaM a^R^(R-RFB)R-^ aM-RFBaM 1 1 X Ta^R-^ + a^RpaM a^R-!aM a"R-1aM_ a^R-HRFB-R)^-1^ a^R-xaw "r a^R-xaM ^R-^Rfb R)R-'5 a&R"1* a^R-^R-1 R^)aM I IT i I (a^RFBaM)(aMR 1&Ai) a^R-xa m + aÂ£R-^ a^R~xa M (a^R-13)a^R~1(RFB ~ R)R~laM (a^R-xaM)2 (C.2) which, in turn, implies that E {FBC } E BC 1 (a^R-1RFBR~1)(a^R-1aM) + (a^R-1RFBR~1aM)(a^R-1) (a^R-xaM)2 (C.3) 110 77 Figure 5.6: PSD estimate of the output of the first system corrupted by white noise with o2 = 0.01 and N = 200. deteriorated. On the other hand, the above knowledge is not necessarily needed by APES1 and hence its performance is not affected. Unlike APES1, MAFI1 does require this knowledge. Yet, the initial condition response is substantially weakened through the frequency selective filtering employed by MAFI1 and hence has little, if any, effect on the amplitude estimates and the system parameter estimates. As N increases, the transient effect becomes less severe, and, consequently, the initial system parameter estimates obtained by using LSEK approach those obtained by APES1 or MAFI1. Example 5.6.2.2 We now consider a second system with A(z-1) = 1 1.9109z-1 + 1.7251z~2 0.7033z-3 + 0.2450z~4, (5.90) and B(z~l) = z~l + 1.0562z~2 + 0.6100z-3 + 0.1912z~4 + 0.0400z5. (5.91) The noise v(n) is an AR(1) signal as in (5.69) except that e(n) is now replaced by a real-valued white Gaussian noise with zero-mean and variance a2 0.01. The probing signal is the same as in the previous example. Figures 5.7(a) to 5.7(c) show the averaged R.MSEs of the a-parameters and the 6-parameters, as well as the number 98 This is especially the case if there are poles in the ARMA model close to the unit circle, since the covariance sequence decays to zero very slowly in such a case. The performances of the AR coefficient estimation for the ARMA signals via the OMYW method by using both the standard and Capon covariance estimates are shown in Figures 6.6(a) to 6.6(d), where N = 256 and M = 32, and where the plots are, again, based on 100 independent trials. The OMYW equations are solved by using the LS method. To reduce the number of figures, the curves are the sum of the MSEs of all AR coefficient estimates versus the number of equations used in the OMYW method. When using the OMYW method, we assume that the AR orders are known. The numbers of equations used in the curves are L = 4,8,16,32,64, and 128. It has been found that the AR coefficient estimates obtained by using the Capon covariance estimates are usually better than those obtained by the sample covariance estimates. Note that for the OMYW estimator based on the sample covariance estimates, the estimation performance may significantly deteriorate with increasing L (a large value of L may be used for lack of a priori information on the ARMA signal in question), whereas the Capon-OMYW estimators performance is much less affected by the increase of L. 6.5.3 MA Model Order Determination For an MA(g) process, the covariances with lags larger than q are all zeros [75] [41]. However, we can expect that the sample covariance estimates for an MA signal will not decay to zero fast enough, while the Capon covariance estimates will usually give f(k) m 0, for k > q, since the Capon method gives better covariance estimates for higher lags. Hence the inference about the type of signal we are dealing with and its order will be easier to make with the Capon method. We consider an MA(4) signal y{n) = e(n) 2.76e(n 1) + 3.809e(n 2) 2.654e(n 3) + 0.924e(n 4). (6.23) 68 5.5.2 The Effect of M All WLS and MAFI amplitude estimators studied in this chapter depend on the choice of M, the subvector length. It is known that as M increases, all of them can better deal with the case of closely spaced sinusoids, but their statistical stability in general decreases [75]. Hence, there is a tradeoff to to kept in mind when choosing M. Note that M should also be smaller than N/2; otherwise, the estimated covariance matrix will be rank deficient. The following example examines the effect of M on the performances of these estimators. LSE1 and LSEK do not depend on M and are thus not considered in this example. The scenario is similar to the first example (AR noise) except that we fix a2 = 10~2, which corresponds to a local SNR of 30.8 dB for the first sinusoid (at fi = 0.1) and 39.2 dB for the third sinusoid (at /3 = 0.3). M is varied from 1 to 16 for all estimators except MAFI1, which requires that M > K (see (5.43) and (5.58)). The MSEs of the amplitude estimates of a3 and, respectively, op, and the corresponding CRBs are shown in Figures 5.4(a) and 5.4(b). As can be seen from these figures, all estimators are sensitive to the choice of M, to a smaller or larger extent. When no sinusoids are close to the one being estimated, such as the third sinusoid in this example, APESl, APESK, and MAFI1 perform quite well for a wide range of M. For the more difficult case as shown in Figure 5.4(b), the choice of M becomes very critical. Based on our empirical experience, a rule of thumb for the choice of M is given in Table 5.1. 5.6 Application to System Identification Consider the linear discrete-time system described by the following equation [68] x(ri) = H(z l)u(n)+v(n), n = 0,1,..., N 1, (5.71) 81 There are basically two Capon PSD estimators, referred to as Capon-1 [16] [44] and Capon-2 [45] herein. We find that, while Capon-2 is capable of finer spectral resolution around the peaks of a spectrum, it is generally a globally poorer spectral estimator than Capon-1. We hence concentrate our interest on Capon-1 for covariance sequence estimation in this chapter. Since the Capon spectra, i.e., the PSD estimates, are shown to be equivalent to AR or autoregressive moving-average (ARMA) spectra, the inversion procedure for computing the exact covariance sequences corresponding to the Capon spectra can be implemented in a rather convenient way. (Note that the calculation of the covariance sequences corresponding to the Capon spectra is an interesting problem by itself.) We also present an FFT-based approximate method to compute the covariance sequences from the Capon spectra. It has been found that the approximate method provides covariance estimates that are almost identical with those obtained by the exact method, while the computational complexity is greatly reduced. Our primary interest is to apply the Capon method as well as the standard approach to ARMA signals. To that end, a few ARMA signals with typical pole and zero locations are studied in our numerical examples. The studies show that considerable improvements are attained by the new Capon method. The Capon covariance estimation method can be readily used in many appli cations. One important class is the ARMA spectral estimation. Since most ARMA spectral estimators rely on the Yule-Walker equations to determine the AR coeffi cients, it may be expected that the better the covariance estimates used, the more accurate the AR coefficient estimates yielded. We examine how the Capon covari ance estimates can be used with the overdetermined modified Yule-Walker (OMYW) method [14] [69] to compute more accurate AR coefficients. We find that the perfor mances of the usage are influenced by the pole and zero locations and, still, generally 26 Theorem 3.3.1 Under Condition C and the additional assumption that ew(n) is circularly symmetrically distributed, the estimation errors in the Capon and APES spectral estimators are asymptotically circularly symmetrically distributed with zero- mean and the following common variance: lim LE (|(cj) a(o;)|2} = (3.56) Proof: See Appendix B. The need to enforce Condition C limits, to some extent, the importance of the previous result. Indeed the assumption made in C is satisfied if (and essentially only if) the signal y(n) has a mixed spectrum and uj is the location of a spectral line. The result of Theorem 1 is relevant to the spectral analysis of a target with dominant point scatterers in the presence of distributed clutter (see [52] and the references therein). In some other applications, however, the main interest is in the continuous component of the spectrum. For example, Condition C does not hold exactly for a target with distributed scatterers since the signature spectrum is continuous at u>. That the previous result is of a somewhat limited interest is also due to its asymptotic character. Indeed, in applications with medium or small-sized data sam ples, the spectral estimators under study have been found to behave quite differently in contradiction with what is predicted by the (asymptotic) result of Theorem 1 (see the numerical examples in Section 5). The finite-sample analysis of the spectral es timators under discussion would consequently be of considerable interest. However, a complete analysis, if possible, appears to be rather difficult at best. A partial one, by making use of a higher-order Taylor expansion technique, is nevertheless feasible. The result is as follows. Theorem 3.3.2 To within a second-order approximation and under the mild as sumption that the third-order moments of ew(n) and ew(n) are zero, the Capon and 6 to the second order. When only a finite number of samples are available, using sam ple covariance sequences implies that the data beyond the observed duration either is zero or repeats itself periodically, which is certainly not a realistic assumption. There have been several attempts in the literature to derive other covariance estimators (see Chapter 2 for some details). It is known that the sample covariance sequence and the data periodogram constitute a Fourier transform pair. It is also known that the pe- riodogram is a statistically inefficient (in fact inconsistent) PSD estimator [75]. This observation suggests that better covariance estimators may be obtained by Fourier inverting better PSD estimators. Since the MAFI PSD estimates are in general more accurate than periodograms, we propose a new covariance sequence estimator by Fourier inverting the MAFI PSD estimates. Specifically, we make use of the Capon PSD and such an approach is referred to as the Capon covariance sequence estimator. The same methodology can be similarly applied to the APES PSD estimates, though the APES covariance estimates are usually similar to the Capon covariance estimates. The reason is that, despite their different performance for complex spectral estima tion, the Capon and APES PSD estimators usually behave quite similarly, especially for continuous spectra (also see the discussions in Section 3.3.2). It is observed that Capon covariance estimates are in general better than sample covariance estimates in terms of mean-squared errors (MSEs). 1.2 Significance and Contributions The main results of this dissertation are as follows. 1. We introduce a new general class of MAFI approaches to spectral estimation. We show that the Capon and APES estimators, though originally not derived in the MAFI framework, are both members of the MAFI class. 2. To within a second-order approximation, we prove that the Capon estimator gives biased (downward) complex spectral estimates whereas the APES method 37 2-D Complex Amplitude Estimation 2-D Complex Amplitude Estimation (a) (b) (c) (d) Figure 3.4: Empirical bias and variance of the 2-D MAFI estimators as the SNRi varies when Ni = N2 = 32 and Mi = M2 = 8. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. 14 they assume on the observed data. By exploiting the AR spectral estimator, Burg proposed a technique that can offer covariance sequence estimates for any desired lag and hence avoids the windowing problem suffered by the sample covariance estimators [10]. However, it was shown in [78] that the Burg covariance estimates are less accurate and more variable than the sample covariance estimates. In Chapter 6, we describe a new method for covariance sequence estimation based on Capon spectra. Estimating structured covariance matrix of the observed data vectors repre sents another type of the covariance estimation problem. For example, the covariance matrix of a stationary complex signal is Hermitian and Toeplitz. However, the sam ple covariance matrix obtained from a finite number of data samples seldom has this structure. Structured covariance matrix estimation is of importance in a variety of ap plications including array signal processing and time series analysis [28]. An intuitive way to obtain structured covariance estimates is to force the desired structures on the sample covariance matrix, a methodology adopted by the Iterated Toeplitz Approxi mation Method (ITAM) [81] [15] [84]. Specifically, ITAM alternatively makes use of rank approximation (via singular value decomposition) and Toeplitzation along the diagonals until convergence is reached. Obviously, such a method is by nature heuris tic and no optimality can be associated with it, though the ITAM covariance matrix estimate is in general closer to the true covariance matrix than the sample covariance matrix in the Frobenius norm sense. Optimum structured covariance matrix estimate may be obtained by maximizing the corresponding likelihood function as considered in [1] [13] [23] [29] [85]. However, since there exists no closed-form solution to the complicated nonlinear ML estimation problem for Hermitian Toeplitz matrices, the ML methods proposed in these studies are iterative and computationally involved, and, moreover, they are not guaranteed to yield the global optimal solution, which to some degree limits the interest in using the ML structured covariance matrix es timate in practical applications. An approximate ML method that makes use of the 23 and 4 PBaM p 1 ^FB 1 / Vg + agj a^Rpg -Rpg (a*g 1 ~~ oaM H-o-l rrtFB (d*g + agj aM R-pg aM 2 aM ^FB a*g + ag R-pBaM + 2a'Ai'^'FBaM'^'FB a*g + ag 1 aH R-1 1 2 dAirLFB a*g + ag (3.39) which gives the following expressions for the matched filter vectors: ^pQaM and FOM aM^ FOaw R-pOaAi + *aM^'FOaM'^'FOS a;*aM-^FC)S'^'FOaM a^R-pQaM (3.40) ,FBM ^PBaAi aW^FBaM R-pBaM + 9aM-^'FBaM'^'FB *g + g - -jCR H D-l M FB &*g + a g R'FBaAi aM-^FBaM (3.41) The previous MAFI filters are in general different from both the Capon and APES filters, since neither the Capon nor the APES filters depend on an estimate of a(oj) which the new MAFI filters need to know. In spite of this fact, in Appendix A, we prove that, for a certain natural choice of a(u>) in (3.36) and (3.37), the following equalities hold true: FOmM = TOaM. (3-42) fbmM = fbaM- (3.43) 115 Proof of Theorem 6.4.1: By making use of Lemma D.0.1 we obtain: M-1M-1 aff(w)ra(w) = alr^ai k=0 1=0 M-1M-1 = EE^-h k=0 =0 M-1 M-1 M1 M-s-1 = E(Er-)e,'" + E( E rM+.)e-< s=0 k=s s=l fc=0 M-1 = E s=-{M-l) with Us defined in (6.12). If = r, then we have Ms1 H-S = r*>fc+*> for s > 0- (D.5) k=0 Taking complex conjugation of both sides yields Ms1 Ms1 (/-.) = E rP+ = E r+*.* fc=0 fc=0 Ml = r*>*-s = (D-6) l=s and the proof is complete. I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Jian Li Chairman Associate Professor of Electrical and Computer Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Jose C. Principe Professor of Electrical and Computer Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. William W. Edmonson Assistant Professor of Electrical and Computer Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. Frederick J. Taylor Professor of Electrical and Computer Engineering I certify that I have read this study and that in my opinion it conforms to acceptable standards of scholarly presentation and is fully adequate, in scope and quality, as a dissertation for the degree of Doctor of Philosophy. David C. Wilson Professor of Mathematics 5 1.1.3 Amplitude Estimation Another problem related to complex spectral estimation is amplitude estima tion for sinusoidal signals where it is assumed that the number and frequencies of the sinusoids are known a priori. The observation noise is stationary and maybe colored. In the case that the noise can be modeled exactly, the Maximum Likelihood (ML) methodology can be used and the ML amplitude estimates are statistically efficient. However, an exact model of the observation noise is usually not available and, more over, ML methods are in general very sensitive to inaccurate model information. As such it may be a better choice to use methods that do not model the noise exactly. In Chapter 5, we describe a relatively large number of amplitude estimators which assume no model except for stationarity for the observation noise. The am plitude estimators can be categorized as three general classes, namely Least Squares (LS), Weighted Least Squares (WLS), and MAFI approaches to amplitude estima tion, which are all asymptotically statistically efficient. For finite length of data samples, however, their behaviors are quite different. We show that under certain circumstances, the MAFI approach to amplitude estimation is equivalent to the WLS approach, and yet the former is more general and includes the latter as a special case. We also show that the MAFI and WLS methods in general give more accurate am plitude estimates than the LS methods. 1.1.4 Covariance Sequence Estimation The fact that covariance (the terms covariance and autocorrelation are used interchangeably with one another in this study) function and Power Spectral Den sity (PSD) are a Fourier transform pair makes the problem of covariance sequence estimation a research topic that is closely related to spectral estimation. Sample covariance sequences have been widely used in signal processing because of its com putational simplicity and its consistency supposing that the given signals are ergodic 52 5.2.2 LSEl.O.ll Since the observation noise v(n) is not modeled, an idea that reduces the computational burden quite a bit is to include K 1 sinusoids in the noise term, and hence estimate only one amplitude at a time. In some signal processing applications, the frequencies {u)k}k=1 may be unknown. A typical way to estimate both {ak}%=l and {uk}Â£=1 would consist of estimating just one amplitude for varying frequency u> and, then, detecting the peaks in the so-obtained spectrum [75] [41] [55]. As such, the assumption made in Section 5.1 that {Luk}k=1 are known a priori may be relaxed when using the one-at-a-time technique. There is a somewhat subtle problem with the above technique: the sum of v(n) and K 1 sinusoids no longer has a finite PSD, and hence one of the previously made assumptions fails. Nevertheless, the idea still works as long as no two sinusoids (that are of interest) are spaced too close to one another, as shown below and later in Section 5.5. The LSE(1,0,1) is easily derived as n=0 which is recognized as the DFT of {^(n)}^1 at ujk. The two estimates in (5.4) and (5.14) will be close to one another if \cok | 7Â§> 1 /N (V k,i,k ^ l) [75]. An analysis of LSE(1,0,1) runs as follows. Without loss of generality, let us consider (5.14) for k = 1. The LSE(1,0,1) estimate of a.\ is given by ai = (7) 1 ^x, (5.15) where T = l eiWi _ ej(N-i)wi (5.16) and where (-)1 denotes the transpose. Taking the expectation of (5.15) yields (5.17) 4 EFFICIENT IMPLEMENTATION OF CAPON AND APES 39 4.1 Introduction 39 4.2 Efficient Implementation of APES 39 4.3 Extension to Capon 43 4.4 Numerical and Experimental Examples 44 4.5 Summary 45 5 AMPLITUDE ESTIMATION 47 5.1 Introduction 47 5.2 LS Amplitude Estimators 50 5.2.1 LSE(1,0, K) 50 5.2.2 LSE(1,0,1) 52 5.3 WLS Amplitude Estimators 54 5.3.1 WLSEL, 0, K) 54 5.3.2 WLSE(L, 0,1) 57 5.4 MAFI Amplitude Estimators 58 5.5 Numerical Examples 62 5.5.1 Estimation Performance versus SNR 63 5.5.2 The Effect of M 68 5.6 Application to System Identification 68 5.6.1 System Identification Using Amplitude Estimation .... 70 5.6.2 Numerical Examples 73 5.7 Summary 78 6 CAPON ESTIMATION OF COVARIANCE SEQUENCES 80 6.1 Introduction 80 6.2 Standard Covariance Estimator and Outlook 82 6.3 Capon PSD Estimator 84 6.4 Capon Covariance Estimator 87 6.4.1 Exact Method 87 6.4.2 Approximate Method 89 6.4.3 Computational Aspects 90 6.5 Numerical Results 91 6.5.1 ARMA Covariance Estimation 92 6.5.2 AR Coefficient Estimation for ARMA Signals 97 6.5.3 MA Model Order Determination 98 6.6 Summary 100 7 CONCLUSIONS 102 7.1 Summary Remarks 102 7.2 Future Work 105 APPENDIXES A PROOF OF (3.42) AND (3.43) 106 B PROOF OF THEOREM 3.3.1 108 v 87 that the large sidelobes do not make any significant contributions to the filter output power so that the filter design criterion is still satisfied; that is, the output power is minimized, while the frequency response at to is one. In all such cases, the Capon filter is not a narrowband filter and hence it calculates an overestimated filter bandwidth. Hence the Capon-2 PSD estimates become highly biased. However, it is interesting to note that Capon-2 does possess higher resolution capability, around the power peaks, than Capon-1. This is illustrated in Figure 6.1(b), which shows the PSD estimates of the same ARMA signal as used in Figure 6.1(a) but with N 32 and M = 10. The Capon-1 estimator cannot resolve the two power peaks this time, while Capon-2, albeit biased, still can. For the preceding reasons we do not consider using Capon-2 for covariance sequence estimation in the sequel. 6.4 Capon Covariance Estimator We describe below how the Capon PSD estimates can be Fourier inverted in a rather convenient manner yielding the Capon covariance sequence estimates. The study of the covariance sequences corresponding to the Capon spectra is an interest ing endeavor by itself, which apparently has not been undertaken in the literature before. We also present an approximate but computationally more efficient method to calculate the Capon covariance estimates from the Capon PSD estimates. 6.4.1 Exact Method Theorem 6.4.1 Let T = {rij} E cW+PxW+i) and let d-M (w) 1 ejuJ then M l aj^(w)raM(w) = Y hseJSUJi s=-(M-1) (6.10) (6.11) 103 complexity. This becomes more evident in 2-D spectral estimation where the amount of computation required by either Capon or APES is much more than by the Fourier- based methods. Efficient implementation schemes were presented in Chapter 4 for both the Capon and APES estimators. It was shown that the efficient implementa tion scheme of Capon or APES reduces the computational complexity substantially, especially when the number of frequency samples are large. In complex spectral estimation, the locations of the spectral components that are of interest to us are in general unknown. Hence, the center frequency of the prefilter must sweep through all frequency bands to obtain the corresponding spec tral estimates. One problem that is closely related to complex spectral estimation is amplitude estimation for sinusoidal signals with known frequencies. Conventional methods for amplitude estimation are based on the technique of Least Squares (LS) fitting. However, LS methods typically perform quite poorly whenever the observa tion noise is colored. By splitting the data vectors into subvectors of shorter length, Weighted Least Squares (WLS) technique can be used for amplitude estimation. We presented in Chapter 5 several WLS amplitude estimators and, interestingly enough, they are closely related to the Capon and APES estimators for complex spectral es timation. We also proposed a generalized MAFI approach to amplitude estimation, where all prefilters corresponding to the different frequencies are simultaneously de termined by defining and maximizing a generalized SNR. It was found that the generalized MAFI approach to amplitude estimation is very general and includes the WLS approach as a special case. We derived an interesting MAFI amplitude estima tors that does not fall into the WLS class. But other MAFI amplitude estimators may exist and are yet to be discovered. Another problem we addressed is covariance sequence estimation. Specifically, we presented in Chapter 6 a new covariance sequence estimator by Fourier inverting the Capon Power Spectral Density (PSD) estimator. We described how to Fourier Ill Next we note that L-l R-FB = [{aaMe3ul1 + e(l)) (aaMe?ul + e(l))H n 1=0 + (aaMej! + e(l)) (a&Mejul + e(l))H = \a\2a.M&M + + *)4 + M(o6H + H) +E[gWe"(') + M0"(Â¡)]. ^ 1=0 (C.4) We also remind the reader of (B.5) and the assumption that eu(n) and ew(n) have zero third-order moments. Since e(l) = JUL-1-l), (C.5) which is like (3.9), we have = = ^L l - ^ 1=0 ^ 1=0 = e~ML-1]J k=0 e-ML- By using these facts, along with (C.3), we can write (C.6) (aMR aM) E {pBC ~ } 1 Â£j(a" R-'aM) 1 (aj^R 1aM)(aSH + a<5)R *<5 +(aÂ£R-15) 1 a^R 1(a*d + ci*d)(a^R xaM) + |(a^R 1aM)(Q!5/ + <5H)R 13lm = \E {a*(a^R-15)2(a^R-1aM) + *(a^R-1)(a^R1)(a^R1aM) +(aR xaw) a^R-^ + (aÂ£fR 1aM)(a^R 1)(R xaM) .-i. C PROOF OF THEOREM 3.3.2 110 D PROOF OF THEOREM 6.4.1 114 REFERENCES 116 BIOGRAPHICAL SKETCH 122 vi 7 is unbiased; we also prove that the bias for the FB Capon is one half that of the FO Capon. These theoretical results, supplemented with the empirical obser vation that Capon usually underestimates the spectrum in samples of practical length while APES is nearly unbiased, are believed to provide compelling rea sons for preferring APES to Capon in most practical applications. 3. We present efficient implementation techniques for the MAFI spectral esti mators. We show that by using such techniques the amount of computation required by the Capon or APES estimator is significantly reduced. 4. The MAFI idea is also extended for amplitude estimation for sinusoidal signals in colored noise. Specifically, we make extensions to the Capon and APES algo rithms to multiple sinusoids with known frequencies. Furthermore, we describe a generalized MAFI approach to amplitude estimation for multiple sinusoids. 5. A new covariance sequence estimator is presented by Fourier inverting the Capon spectral estimates. The Capon covariance sequence estimates are shown to be more accurate than the conventional sample covariance sequence esti mates. 1.3 Organization of the Dissertation The dissertation is organized as follows. Chapter 2 gives a literature survey of such topics as filterbank approaches to spectral analysis, the Capon spectral estima tor and its applications, and covariance estimation. In Chapter 3, we first introduce the MAFI approach to complex spectral estimation. We next show that the Capon and APES estimators are both members of the MAFI approach, followed by com putational and statistical analyses of the MAFI spectral estimators. Extensions to the 2-D case are also included. Chapter 4 addresses the implementation issue of the MAFI spectral estimators. Chapter 5 discusses amplitude estimation for sinusoidal 76 (a) Computational Complexity Data Length N (c) (b) Figure 5.5: Averaged RMSEs and the number of flops versus N for the first system when the observation noise is white (a2 = 0.01) and M = 20 for APESl and MAFI1. (a) RMSE of a-parameters. (b) RMSE of 5-parameters, (c) Number of flops. 3 (c) (d) Figure 1.1: Synthetic Aperture Radar (SAR) images of a simulated MIG-25 airplane obtained by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES. 4 resulting filters are matched filters and we refer to this class of approaches to spectral estimation as MAtched Fllterbank (MAFI) approaches. We show that both Capon and APES are members of MAFI approaches, although none of them was originally derived in the MAFI framework (see [16] [52] for their original derivations). MAFI approaches to spectral estimation may also be used to devise new spectral estimators. Even though we show in Section 3.2.3 that a reasonable implementation of a seemingly novel MAFI spectral estimator reduces back to APES as well, it remains an open issue whether other interesting MAFI spectral estimators exist. The MAFI interpretation also provides insights into the Capon and APES estimators and the relationship between them. Specifically, in the framework of MAFI approaches, we show by means of a higher-order expansion technique in Sec tion 3.3 that the 1-D (one-dimensional) Capon estimator indeed underestimates the true complex spectrum while the 1-D APES is unbiased (to within a second-order approximation); we also show that the bias of the FB Capon is one half that of the FO Capon. Furthermore, we show that these results can be extended to 2-D (two-dimensional) Capon and APES estimators. 1.1.2 Efficient Implementation For 2-D applications, a major concern of using Capon or APES is their com putational loads which in general are much heavier than those of the traditional Fourier-based approaches. For a SAR image of size 128 x 128 formed from a 32 x 32 data matrix, as those in Figure 1.1, the number of flops required by Capon and APES implemented in the conventional way are 2.3 x 104 and, respectively, 3.0 x 104 times that of those by the Fourier-based methods. It should be mentioned that a SAR image of 128 x 128 is relatively small. As the size of the image increases, the amount of computation by Capon or APES increases drastically. In Chapter 4, we address the issue of how to efficiently implement Capon and APES for spectral estimation. 73 Remark-. According to the Extended Invariance Principle (EXIP) [76], the estimates of {a,i,bj} obtained by minimizing (5.82) achieve the CRB asymptotically, and hence they have a better asymptotic accuracy than the OEM estimates whenever v(n) is colored. It also follows from this observation that in the case of AT = p + q, the estimates obtained from (5.85) are asymptotically efficient. This latter result (of a somewhat limited interest, due to the requirement that K p + q) was first proved in [40] in a much more complicated way. 5.6.2 Numerical Examples The following examples assume that p and q are known to facilitate perfor mance comparison. It is reasonable to do so since both OEM and the proposed method use similar techniques to determine the model orders. Also, we adopt the strategy to choose the p + q largest {or*} in Step 2 of the proposed method. Example 5.6.2.1 The system considered in this example is given by (5.73) with A^z-1) = 1 1.6019Z-1 + 0.9801z"2, (5.86) and B(z~1) = z-1 + 0.24722-2 + 0.1600z3. (5.87) The probing signal is given by u(n) = 2 cos(27r0.05n) + 2 cos(27r0.15n) 4-2 cos(27r0.25n) +2 cos(27r0.35n) + 2 cos(27r0.45n), n = 0,1,..., N 1. (5.88) We consider using a real-valued probing signal because this is the usual case in practice. (A subtle question arises as the amplitude estimation techniques discussed in the previous sections all assume that the sinusoids are complex-valued. One might impose certain conjugate symmetry constraint and derive similar techniques that are specifically tailored for real-valued sinusoidal amplitude estimation so that, if o6 drawback of Q in (5.34). In the following we try to circumvent this need in two different ways. First, we show a way to simplify the WLSE(L, 0, K) that uses (5.29) with (5.34). From (5.34), we have that RQ_1A = APA^Q^A + A = AI\ (5.35) where r = PA^Q^A + I*. (5.36) For sufficiently large N and M, T is approximately diagonal since AHQ~1A is so (see, e.g., (5.12)). Consequently, Q_1A; = R_1ArD; R-^Djr = RxA;r. Inserting (5.37) into (5.29) yields (observe that VH cancels out) n -1 rr (5.37) ex EifR-'A, L/=o L-l U=o (5.38) which, unlike using (5.29) with (5.34), does not require any initial estimate of {ex*,}^!. The amplitude estimator in (5.38) can be interpreted as an extension of the Capon algorithm in [16] [44] to multiple sinusoids. A different estimate of Q can be obtained as described next. Observe that K APAh = ^2[aka{ujk)}[aka{uk)]H = K k=1 k=1 where a(cj) = 1 e (5.39) (5.40) We can use the vectors {/3k}Â£=l introduced above to rewrite (5.27) as K y(0 = k=1 (5.41) 38 2-D Complex Amplitude Estimation 2-D Complex Amplitude Estimation (a) (b) (c) (d) Figure 3.5: Empirical bias and variance of the 2-D MAFI estimators as the filter length, M = M\ = M2, varies when N\ = N2 = 32, and SNRi = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude. one can see that R is persymmetric. Similarly, Q is also persymmetric. The forward-backward sample covariance matrix takes the form: 29 rFB = 2(RFO + rB0)> (3.70) where Rpo and Rbo denote the sample covariance matrices of {y(1, 2)} and {y(h, Z2)}, respectively, given by Rpo = R-BO = 1 L\L2 L\ \ L/2~ 1 Y Y y(h,h)yH{h,i2), h= 012=0 1 LiL2 EEWuWduk). ll= 0 Â¡2=0 (3.71) (3.72) By making use of (3.69), one can see that Rfb is also persymmetric. Let HWlA,2 denote the impulse response of an x M2 2-D FIR filter, and let h-o;1,i2 vecfH^j^]. (3.73) Like in the 1-D case, the impulse response of the matched filter is given by , Q-1(aJi,aÂ¡2)aMi,M2(wi> w2) XWl,J2 h aMi ,m2 (wi ) w2)Q 1 (aÂ¡i, oÂ¡2)aMl ,m2 (wi ^2) (3.74) Note that hwi, U2aMi,M2(wi ,V2) I- (3.75) The LS estimates of u>2) obtained by using only the forward data vectors and by using both the forward and backward data vectors are given by (similarly to (3.21) and (3.22) in the 1-D case) Fo(wi,aÂ¡2) = h"g(w), (3.76) and -j(Ni-l)uÂ¡ie-j(N2- (3-77) 92 Processes ARMA1 ARMA2 ARMA3 ARMA4 ai -1.1824 -1.7351 -0.2000 -0.2000 0-2 0.6651 1.7829 0.0400 0.0400 a3 -0.0895 -0.9616 0.0000 0.0000 <24 0.0049 0.3969 0.0000 0.0000 b\ -0.2000 -0.2000 -1.1824 -1.7351 b2 0.0400 0.0400 0.6651 1.7829 ^3 0.0000 0.0000 -0.0895 -0.9616 b4 0.0000 0.0000 0.0049 0.3969 a2 1.0000 1.0000 1.0000 1.0000 Table 6.2: The ARMA processes used in the numerical simulations. 6.5.1 ARMA Covariance Estimation First we comment on the generation of the ARMA signals. To eliminate the initial transient caused by improper initialization of the ARMA system we proceed as follows. Given the coefficients of an ARMA(p,q), we can determine the covariance matrix (see e.g., [75]) r(0) r(l) r(p 1) r*(l) r(0) : I r(l) r*(p 1) r*(l) r(0) (6.20) We use C1/2e as the initial condition for the ARMA system, where e G Cpxl is a zero-mean Gaussian random vector with identity covariance matrix. Four ARMA signals are chosen for our study. In the selection of these ARMA signals, efforts have been made to make them representative of a large class of ARMA signals. The coefficients of the ARMA signals are listed in Tables 6.2. The pole and zero diagrams are shown in Figures 6.2(a) to 6.2(d). Figures 6.3(a) to 6.3(d) give the PSDs of the ARMA signals, while Figures 6.4(a) to 6.4(d) show the corresponding covariance plots for these signals. 31 Based on the 2-D extensions described above, it is not difficult to see that all the results of the previous section also hold true for the 2-D Capon and APES estimators. Indeed, the proofs for the 2-D estimators follow a similar pattern to those for the 1-D case shown in Appendixes B and C. 3.5 Numerical Examples In the following, we study the Capon and APES complex amplitude estimates in a number of cases of interest. For both the 1-D and 2-D examples given below, we compare the performance of the forward-only Capon and APES as well as the forward-backward Capon and APES, which are, for simplicity, referred to as FCapon, FAPES, FBCapon and FBAPES, respectively. 3.5.1 1-D Complex Spectral Estimation The 1-D data used in the examples consists of a sum of 15 complex sinusoids, with the real and imaginary parts shown in Figures 3.1(a) and 3.1(b), respectively, corrupted by a zero-mean complex white Gaussian noise. The data length is chosen as N = 64. In what follows we are interested in the bias and variance properties of the estimators under study. The bias and variance results shown below correspond to the frequency of the first sinusoid and they are obtained from 100 independent realizations. We begin by studying the performance of the estimators as the signal-to-noise ratio (SNR) varies. The SNR for the kth sinusoid is defined as SNRfe = 101og10-^h- (dB), (3.87) 0e(^/c) where is the complex amplitude of the kth sinusoid and Pe{u)k) is the spectral density of the additive noise at frequency 04. The filter length is chosen as M = 15. The real and imaginary parts of the bias are shown in Figures 3.2(a) and 3.2(b), respectively, as a function of SNRi. As seen from these figures, FOA and FBA are 33 1-D Complex Sinusoids 1-D Complex Sinusoids (a) (b) Figure 3.1: The 1-D complex amplitude of the sum of 15 sinusoids used in the simulations, (a) Real part; (b) Imaginary part. resolution as well as the best statistical properties in terms of bias and variance. The previous examples also show that FOA and FBA perform similarly in terms of bias and variance properties for the frequency of interest. To compare the computational complexities of the estimators under study, we count the flops required by each of them for the case where N = 64, M = 24, and the complex spectra are evaluated at 256 equally spaced points. The flops required by FOC and FBC are approximately the same, whereas the flops needed by FOA and FBA are, respectively, 1.08 and 1.41 times of that by the Capon estimators. 3.5.2 2-D Complex Spectral Estimation As was mentioned in Section 3.4, the 2-D Capon and APES estimators be have rather similarly to their 1-D counterparts. Since the problems encountered in applications such as synthetic aperture radar imaging are concerned with 2-D com plex spectral estimation, we include a couple of 2-D numerical examples here. The data employed consists of three 2-D sinusoids corrupted by a 2-D zero-mean complex white Gaussian noise, with N\ = N2 = 32. The sinusoids are located in the frequency domain at (0.2, 0.2), (0.25, 0.25) and (0.4, 0.7) and their amplitudes are e-J7r'/4, eJ7r,/4 45 FFT, 2-D windowed FFT, 2-D Capon, and 2-D APES are shown in Figures 4.1(a) to 4.1(d), respectively. Again, Capon and APES outperform the FFT methods. The number of flops required by the intuitive ways of implementing Capon and APES are, respectively, about 38 and 32 times of those required by our efficient ways of implementing them. If we increase the size of the image to 512 x 512, the ratios of the needed MATLAB flops between the intuitive ways and our new ways of implementing Capon and APES are 86 and 73, respectively. 4.5 Summary This chapter addresses the implementation of the Capon and APES spectral estimators. The amount of computation required by APES is shown to be about 1.5 times that required by Capon. By using a technique proposed in this chapter, the computational complexities of Capon and APES are significantly reduced. This work is dedicated to my wife, Hong. 118 [32] L. J. Griffiths and C. W. Jim. An alternative approach to linearly constrained adaptive beamforming. IEEE Transactions on Antennas and Propagation, AP- 30:27-34, January 1982. [33] I. J. Gupta. High-resolution radar imaging using 2-D linear prediction. IEEE Transactions on Antennas and Propagation, 42(l):31-37, January 1994. [34] E. J. Hannan and B. Wahlberg. Convergence rates for inverse Toeplitz matrix forms. Journal of Multivariate Analysis, 31:127-135, October 1989. [35] S. Haykin, editor. Array Signal Processing. Prentice-Hall, Englewood Cliffs, NJ, 1985. [36] M. Honig and U. Madhow. Blind adaptive multiuser detection. IEEE Transac tions on Information Theory, 41(4):944-960, July 1995. [37] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, Cambridge, 1985. [38] Y. Hua. High resolution imaging of continuously moving object using stepped frequency radar. Signal Processing, 35(l):33-40, January 1994. [39] A. Jakobsson, T. Ekman, and P. Stoica. Capon and APES spectrum estima tion for real-valued signals. Proceedings of the 8th IEEE DSP Workshop, Bryce Canyon, August 1988. [40] P. V. Kabaila. On output-error methods for system identification. IEEE Trans actions on Automatic Control, 28(l):12-23, January 1983. [41] S. M. Kay. Modern Spectral Estimation: Theory and Application. Prentice-Hall, Englewood Cliffs, NJ, 1988. [42] S. M. Kay and S L. Marple, Jr. Spectrum analysis a modern perspective. Proceedings of the IEEE, 69(11): 13801419, November 1981. [43] S. B. Kesler, editor. Modern Spectrum Analysis II. IEEE Press, New York, 1986. [44] R. T. Lacoss. Data adaptive spectral analysis methods. Geophysics, 36(4):661- 675, August 1971. [45] M. A. Lagunas and A. Gasull. An improved maximum likelihood method for power spectral density estimation. IEEE Transactions on Acoustics, Speech Signal Processing, ASSP-32(1):170-173, February 1984. [46] M. A. Lagunas, M. E. Santamaria, A. Gasull, and A. Moreno. Maximum like lihood filters in spectral estimation problems. Signal Processing, 10(1): 19-34, January 1986. [47] H. Li, J. Li, and P. Stoica. Performance analysis of forward-backward matched-filterbank spectral estimators. IEEE Transactions on Signal Process ing, 46(7):1954-1966, July 1998. [48] H. Li, P. Stoica, and J. Li. Capon estimation of covariance sequences. Circuits, Systems, and Signal Processing, 17:29-49, January 1998. 43 and e(ujuuj2) via 2-D FFT, then using Equations (4.18), (4.19), and (4.20) to deter mine a^i M2Q-1t/>, 'tp11 Q~lip, and a^1>M2Q~1&mum2, respectively, and finally using (4.12) to obtain APES^i)^2)- The structure of (3.53) is similar to that of (4.12). However, the amount of computation required by the former is much larger than that by the latter. The reason is that, even though ?/>(u>i,u>2) and ^(uj \,u2) can be obtained by 2-D FFT, for each (coi,tu2) pair, we have to compute the additional matrix-vector products R-1/2t/>(o>i,w2) and R_1/2'0MllM2(a;i,^2) (recall that R_1/2 e CMlM2XMlM2, %j){ui,u2) and ip(u>i,u2) CMlMiXl) to obtain /x2(w 1^2) and u2). On the other hand, by computing D and E first (which are computed only once), we bypass calculating such matrix-vector products and save a large amount of computation. The larger the number of samples in the 2-D frequency domain, the more the amount of computation we will save. These discussions also apply to the implementation of Capon. 4.3 Extension to Capon Note that APES becomes Capon when Q^,^) is replaced by R. Hence the efficient implementation of Capon can readily be achieved by modifying (4.12) as follows: bT(wi,W2)d(Â£Ji,W2) . CaponKwa) LlÂ£2 ||b(Wl>W2)||2 More specifically, the efficient implementation of Capon is by using (4.13) and (4.14) to calculate b(aq, o>2) and d(uq, u>2), respectively, and then using them in (4.27). Since the amount of computation required to calculate b(a>,cD) in (4.13), d(ui,u)2) in (4.14), or e(a)i,cu2) in (4.15) is approximately the same and calculating APES O-A >^2) by (4.12) and dQap0n(wi, lo2) by (4.27) are much less involved than obtaining b(ui,u2), d(uq,u;2), and e(ui,u)2), the total amount of computation required by APES is about 1.5 times of that required by Capon, as verified by the numerical and experimental examples in Section 4.4. LIST OF TABLES 5.1 Choice of M for the WLS and MAFI amplitude estimators 70 6.1 Comparison of the computational burdens of the standard and Capon methods with N = 256 and 512 91 6.2 The ARMA processes used in the numerical simulations 92 vii 84 than the latter [44] [75]. Another reason that we consider the Capon PSD estimator is that it does not exhibit the so-called correlation matching property [75] [41]; that is, the inverse Fourier transform of the Capon PSD estimates does not yield the same covariance sequences used to obtain the Capon PSD estimates. This fact allows us to obtain a new covariance estimator from the Capon spectra. 6.3 Capon PSD Estimator We have derived in Section 3.2.1 the Capon filters and Capon amplitude and phase spectra. In this section, we derive expression for the Capon PSD estimates. The Capon filter is rewritten below for easy reference (see (3.28)): R-1aM(w) ^ Capon 1AcJ The filter output power is given by Â£{|hfy(Â¡)|2} =h"Rh, (6.2) 1 (6.3) a^(w)R-1aM(u>) Let Â¡5 denote the bandwidth of the filter given by (6.2). Then the Capon PSD estimate has the form ir f\uH,-.nW2\ i 0(u>) {ihJyWl2} (6.4) P /3a^(cu)R-1aM(u;)' Since the (equivalent) time-bandwidth product is equal to unity, one way is to choose P as the reciprocal of the temporal length of the Capon filter; that is 1 0 = M' (6.5) By choosing the filter bandwidth as given by (6.5), we obtain the so-called Capon-1 PSD estimator [44] [75]: M Capon-1: <Â¡>(u) = , (6-6) a" (cu)R-1aM(a;) where we have replaced R by the sample covariance matrix R. We may use the forward-only sample covariance matrix, but we prefer using the forward-backward 96 ARMA1 (N=256, M=51,100 realizations.) ARMA2 (N=256, M=51, 100 realizations.) (a) (b) ARMA3 (N=256, M=51,100 realizations.) ARMA4 (N=256, M=51,100 realizations.) (c) (d) Figure 6.5: Covariance sequence estimation with N = 256 and M = 50. The mean- squared errors (MSEs) of the covariance estimates, normalized with respect to r(0), are based on 100 independent realizations, (a) ARMA1; (b) ARMA2; (c) ARM A3; (d) ARMA4. 41 It follows that (4.1) can be rewritten as APES(W1.W2) lmum2 Q [LiLi if) Q &muM2Q laMi,M2 + aM!,M2Q ? (4-12) Since R is Hermitian and positive definite, we can obtain an upper triangular matrix C by Cholesky factorization such that R_1 = Cri^C-1)^ [30]. Let br(wi,w2) = aÂ£ (wi.waJC \ (4.13) d(o>i,w2) Dai2(wi,w2), (4.14) and e(wi,w2) Ea1|Â£j(wi,W2), (4.15) where D = (C-1)HZ, (4.16) and E = (C-1)HZ. (4.17) We have [53] aMi ,M2 Q = aMi,M2R~ i^/) "b b^1 (iJi, cu2)d(iu2, cj2) d- bT(cJi, tu2)e(q>i, cu2)e//(n;1, a>2)d(a>i, oj2) LiL2 ||e(cji,iu2)||2 , (4.18) ipH Q 1tp = R V> + TTfX L\L2 V R-1^ ||d(wi,a;2)||2+ |d7i(a;i, a;2)e(a;i, w2) | L\L2 ||e(Â£Ji,u;2)|| 2 2_ > (4.19) Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy. MATCHED-FILTERBANK APPROACHES TO SPECTRAL ANALYSIS AND PARAMETER ESTIMATION By Hongbin Li May 1999 Chairman: Jian Li Major Department: Electrical and Computer Engineering MAtched-FIlterbank (MAFI) estimators represent a general class of methods that make use of a set of matched filters for various estimation purposes. This disser tation investigates using MAFI approaches for complex spectral analysis, amplitude estimation for sinusoidal signals, and covariance sequence estimation. For complex spectral analysis, we show that the widely used Capon and the re cently introduced APES estimators are both members of the MAFI approach, though neither was originally derived in the MAFI framework. We prove that, to within a second-order approximation, Capon is biased downward whereas APES is unbiased, and that the bias of the forward-backward Capon is one half that of the forward-only Capon. We also show that Capon and APES are of similar computational com plexities and both are more involved than most Fourier-based methods, especially for 2-Dimensional (2-D) data. Efficient implementation schemes which substantially xi This dissertation was submitted to the Graduate Faculty of the College of Engineering and to the Graduate School and was accepted as partial fulfillment of the requirements for the degree of Doctor of Philosophy. May 1999 Winfred M. Phillips Dean, College of Engineering Mihran J. Ohanian Dean, Graduate School 66 Gaussian noise. The SNR is defined in the same manner as in (5.70). Figures 5.3(a) and 5.2(b) show the MSEs of the amplitude estimates of 0:3 and, respectively, aq, and the corresponding CRB as the SNR increases. As one can see, the APES1, APESK, and MAFI1 estimates of a3 are again very close to the CRB; whereas for an, all suboptimal methods suffer from some performance loss as compared to the optimal LSEK, and yet the differences between LSEK and MAFI1 for all SNRs considered here are fairly small. A brief summary based on the previous study is as follows. APES1 is recom mended in applications where it is known a priori that no two sinusoids are closely spaced (see, e.g., the application discussed in the next section), or when the closely- spaced sinusoids are of no interest. The reason to prefer APESl to APESK or MAFI1 in such cases is that the former is more flexible than the latter two since APESl does not necessarily require the knowledge of the sinusoidal frequencies. In terms of com putational cost, APESl and MAFI1 are similar to one another and both are simpler than APESK. When it is desired to estimate closely spaced sinusoids in colored noise, however, MAFI1 may be preferred. In general, we do not recommend the use of Caponl since it has a computational complexity similar to that of APESl but is biased. Although we did notice that CaponK gives close-to-CRB performance at very low SNRs, in most cases of interest, other methods like APESl or MAFI1 may be preferred. LSEK is statistically efficient and may be preferred when the observation noise is white; in cases where the white noise assumption is invalid, it is preferable to use APESl or MAFI1. LSE1 gives comparatively rather poor estimation accuracy but is computationally quite simple. The performance differences stated so far occur only when N is relatively small. As N increases, all methods tend to the CRB, in dependent of the noise correlation. Hence, when N is sufficiently large, LSE1 should be preferred because of its computational simplicity. 12 in great length, vve describe in the following a few typical applications of the Capon estimator. An interesting application of Capons method is beamforming. Beamforming is used in conjunction with an array of sensors to provide a versatile form of spa tial filtering. The objective of beamforming is to estimate the signal arriving from a desired direction in the presence of noise and interfering signals. If the desired and in terfering signals occupy the same (temporal) frequency band, then temporal filtering cannot be applied to distinguish signal from interference. However, since the desired signal and interfering signals typically originate from different locations, such spatial diversity can be exploited to separate signal from interference using a spatial filter. In 1972, Frost made use of a linear constrained optimization technique and introduced an adaptive beamformer [27], referred to as the LCMV (Linearly Constrained Mini mum Variance) beamformer in the array signal processing community. The basic idea of the LCMV beamforming is to constrain the response of the beamformer so that signals from the direction of interest are passed with specified gain while minimizing the output power due to interfering signals and noise arriving from other directions. One would immediately notice the similarity to the constraints adopted by Capon. Indeed, the LCMV beamformer is a direct extension of the temporal Capon filter to the spatial domain. Among the so-called statistically optimum beamformers, LCMV is perhaps the most popular one since it needs no auxiliary channels as required by the Multiple Sidelobe Canceller (MSC) [2], and, unlike the class of optimum beam- formers proposed by Widrow et al. [83] which require reference signals, it is blind. A useful structure for LCMV implementation is the Generalized Sidelobe Canceller (GSC) [32]. GSC represents an alternative formulation of the LCML beamformer which changes the constrained optimization problem of LCML to an unconstrained one. The unconstrained nature lends GSC to adaptive implementation more readily than the original LCMV beamformer and hence GSC is the one used more often 36 and 0.7eJ7r/4, respectively. The bias and variance for the amplitude estimate of the first 2-D sinusoid are obtained from 100 independent realizations. The SNR for the kth 2-D sinusoid is similarly defined as in (3.87). The bias and variance of the four estimators under study versus SNRi are shown in Figures 3.4(a) to 3.4(d), respec tively, where Mi = M2 = 8. Figures 3.5(a) to 3.5(d) show the statistical results as the 2-D FIR filter length varies, where SNRi is fixed at 20 dB. We assume in Figures 3.5(a) to 3.5(d) that Mi = M2. As seen from these plots, the performance of the 2-D MAFI estimators indeed resembles that of their 1-D counterparts and, therefore, we refer the readers to the 1-D examples for comments. 3.6 Summary This chapter discusses using the MAFI approach for complex spectral estima tion. The Capon and APES estimators are shown to be members of the MAFI class. By using a higher-order expansion technique, it is proved that to within a second- order approximation Capon is biased (downward) while APES is unbiased, and that the bias of the forward-backward Capon is one half that of the forward-only Capon. It is also shown that the above conclusions carry over to the 2-D MAFI estimators as well. Since computationally APES is only slightly more involved than Capon, the preference of APES to Capon in practical applications follows logically because of the better statistical properties associated with the former. 21 matrix inversion lemma [30], one can see that the second term in (3.27) has no influence on the hw in (3.17). Hence, when (3.27) is substituted into (3.17), the matched filter reduces to the Capon filter [16] [44]: R aj^(u;)R-1aM(a;) (3.28) Observe that Q(tu) is persymmetric for either FOC or FBC. By substituting (3.28) into (3.21) or (3.26), we obtain the Capon estimate of 3.2.2 APES Filter ^CaponC^) aj^(o;)R-^(w) (3.29) Ignoring the fact that am(lo) is known, we obtain the LS estimate of the vector a(w)aM(w) in (3.4) as [a(u)a.M(uj)] = g(u>). (3.30) Inserting (3.30) into (3.27) along with Rpo substituted yields the FOA estimate of QM: QfOaM = R-FO g(w)gH(w). (3-31) A persymmetric estimate of Q(u) can be obtained by using both the forward and backward data vectors: QfbaM = ^ [QfOaM + JQfOa(w)J = R-FB G(w)G/i(u;), (3.32) where G(w) = V2 g(cu) g(u) (3.33) and we have used the fact that Jg*(w) = e^L~^g(oj). Hence by (3.17), we obtain the the APES filter [52]: ,APES Q^l(^)aM(w) h aw(w)Q -1MaM(w) (3.34) 30 where g(wi,w2) = y{h,h)e J'(wi,1+W2ia), 1 2 i1=0 /2=o g(wi,w2) 1 L1 1 Z/2 1 L\L- y(/i>/2)e" j(wii+W2/2) (3.78) (3.79) ;1=o 2=o Since Q is persymmetric, (3.77) can be written as Fb(wi> ^2) h5liW2g(wi, w2). The Capon method estimates the noise covariance matrix as (3.80) Qcapon^i)^2) R |a(wi,W2)|2aAi1)M2(^1,^2)a^-liM2(a;i,w2), (3.81) where Â¡Â¡(cjx,^) denotes some estimate of g:(cji,j2), and R denotes either Rpo or Rpg, which correspond to 2-D FOC or FBC. Thus, the Capon estimate of a(u>i,cj2) is obtained as ^Capon (wl) ^2) aMuM2(ui,V2)R Xg(wi,t^) aMi,M2(Wli(J2)R W2) The FO and FB APES estimates of Q(u>i,a;2) take the form: QfOA(W1)W2) = RfO g(^l;^2)g(^l)^l), (3.82) and where Qfba(w1i ^2) Rfb G(wi, cj2)Gh(cji, uji), G(cji,cj2) y/2 g(uJi,W2) g(idi,U>2) (3.83) (3.84) (3.85) Hence we obtain APES (^l) ^2) as aM1,M2(Wl>a;2)Q 1(Wl^2)g(^l,W2) APEs(wi)w2) = aMi ,M2 (w1 > w2)Q 1 (^1 > ^2)aMi ,M2 (t^l, ^2) where Q(u,i,cj2) denotes either QfOa(wi>cj2) or Qfba(wi> ^2)- (3.86) CHAPTER 4 EFFICIENT IMPLEMENTATION OF CAPON AND APES 4.1 Introduction In the previous chapter, we rederived the Capon and APES spectral estimators using the MAFI approach. In the MAFI framework, a number of statistical properties of Capon and APES were obtained. However, we did not address very carefully the implementation of the Capon and APES estimators. Equations (3.46), (3.47), (3.52), and (3.53) give the intuitive ways of implementing Capon and APES. However, such intuitive implementations are computationally expensive, especially for 2-D spectral estimation from 2-D data sequences. In this chapter, we study how to implement Capon and APES efficiently. For simplicitys sake, we only consider 2-D spectral estimation since 1-D spectral estimation is a special case of the former; we also only consider forward-backward Capon and APES since they are more often used than their forward-only counterparts. 4.2 Efficient Implementation of APES First we rewrite the 2-D forward-backward APES estimator as aMuM2(vi,u>2)Q~l(ui,u)2)ip(uJi,U2) APEs(w>w) = L\L2a.Ml iM2 (cji a;2) Q 1 (uq, ui2)aMl )A2 o>2) where Q(cu,u;) = R - L\L2 ^ H ii r/>(ai,))tj> (w,Co) + '0(w,w)t/> (w,il>) H , (4.1) (4.2) with R denoting the forward-backward sample covariance matrix given by (3.70), #(!,<*) = (4.3) 1=0 2=0 39 APPENDIX A PROOF OF (3.42) AND (3.43) Using the MAFI filters given in (3.40) and (3.41) for spectral estimation re quires an initial estimate of a{to). However, this turns out unnecessary. Proof of (3.42): Inserting (3.40) into (3.21) yields poaM) = a^RioS + (a^RFO^Xg^Rpog) la^Rpogl2 > (A-1) from which we obtain (by assuming that the os in the right-hand and left-hand sides are identical, i.e., both are o:fom): FOM iH lM Rpog aMRFOaM (a^Rpo ) (g W Rpog) la^RpoSl2 Next we rewrite the FOA spectrum (see (3.35)) as follows: aÂ£(Rp0 ggH)1g (A.2) FOA aÂ£(RF0 ggH)_laM \H R FO + i g^R-iog ) g R-gg^R-\ F0 l g"Ri,g ) M Rp + ,-1 s RFQg (A.3) aMRFOaM (aMR'FOaM)(s//Rpog) ~ la^Rpogl which is identical to (A.2). Hence the equality in (3.42) is proved. Proof of (3.43): Substituting (3.41) into (3.26), and after simple manipula tions, we obtain aM%BaM ~ ^(g/7RFBg)(aMRFBaM) + q la^rRpgg|2 ApBM + J(aMRFBg)(g//Rpj3aM) ~ F(g/7RFBg)(aMRFBaM) ^FBM a^Rpgg, (A.4) 106 8 signals from observation corrupted by colored noise. The MAFI concept introduced in Chapter 3 is extended for amplitude estimation. In Chapter 6, we investigate covari ance sequence estimation using the Capon spectral estimates. Finally, we summarize this work and outline future work in Chapter 7. REFERENCES [1] T. W. Anderson. Asymptotically efficient estimation of covariance matrices with linear structure. The Annals of Statistics, 1 (1): 135141, 1973. [2] S. P. Applebaum and D. J. Chapman. Adaptive arrays with main beam con straints. IEEE Transactions on Antennas and Propagation, 24:650-662, Septem ber 1976. [3] D. A. Ausherman, A. Kozma, J. L. Walker, H. M. Jones, and E. C. Poggio. Developments in radar imaging. IEEE Transactions on Aerospace and Electronic Systems, 20(4):363-400, July 1984. [4] G. R. Benitz. Preliminary results in adaptive high definition imaging for sta tionary targets. MIT Lincoln Laboratory Project Report AST-34, November 1993. [5] R. B. Blackman and J. W. Tukey. The Measurement of Power Spectra from the Point of View of Communications Engineering. Dover, New York, 1958. [6] L. E. Brennan, J. D. Mallett, and I. S. Reed. Adaptive arrays in airborne MTI radar. IEEE Transactions on Antennas and Propagation, 24(5):607-615, September 1976. [7] L. E. Brennan and I. S. Reed. Theory of adaptive radar. IEEE Transactions on Aerospace and Electronic Systems, 9(2):237-252, 1973. [8] T. B. Bronez. On the performance advantage of multitaper spectral analysis. IEEE Transactions on Signal Processing, 40(12):2941-2946, December 1992. [9] J. P. Burg. Maximum entropy spectral analysis. Proceedings of the 37th Meeting Society of Exploration Geophysicists, Oklahoma City, OK, October 1967. [10] J. P. Burg. A new analysis technique for time series data. NATO Advanced Study Institute on Signal Processing with Emphasis on Underwater Acoustics, Enschede, the Netherlands, August 1968. [11] J. P. Burg. The relationship between maximum entropy and maximum likelihood spectra. Geophysics, 37(2):375-376, April 1972. [12] J. P. Burg. Maximum Entropy Spectral Analysis. Ph.D. Dissertation, Stanford University, 1975. [13] J. P. Burg, D. G. Luenberger, and D. L. Wenger. Estimation of structured covariance matrices. Proceedings of the IEEE, 70(9):963-974, September 1982. [14] J. A. Cadzow. Spectral estimation: An overdetermined rational model equation approach. Proceedings of the IEEE, 70(9):907-939, September 1982. 116 CHAPTER 5 AMPLITUDE ESTIMATION 5.1 Introduction Consider the noise-corrupted observations of K complex-valued sinusoids [71] [74] K x(n) = '^2akejWkn-\-v(n), n = 0,1,... N 1, (5.1) k=1 where aÂ¡t denotes the complex amplitude of the kth. sinusoid having frequency Uk, N is the number of available data samples, and v(n) is the observation noise which is complex-valued and assumed to be stationary (and possibly colored) with zero-mean and finite unknown Power Spectral Density (PSD) We assume that {<>k}k=i are known, with u>k ^ toi, for k ^ l. The problem of interest is to estimate {ak}^=1 from the observations {^(n)}^^1. In this chapter, we describe a relatively large number of methods for solving this problem. Section 5.2 discusses least squares (LS) methods. LS methods are widely used for amplitude estimation because they are simple and easy to implement. If we restrict ourselves to estimating only one amplitude at a time, then the LS method reduces to the Discrete Fourier Transform (DFT) of the data at the frequency of the desired sinusoid and is computationally more efficient than the LS method that estimates K amplitudes simultaneously. Moreover, estimating one amplitude at a time does not necessarily require exact knowledge of the number of sinusoids in the data and of the frequency location of each sinusoid, which is a desired property in some applications. The disadvantage, however, is that using this one-at-a-time technique in general gives rather poor amplitude estimates when some sinusoids (that 47 88 where min(M+s1,M1) /A ^ r k,k-s- (6-12) fe=max(0,s) IfT is Hermitian, then is = ji*_s. Proof: See Appendix D. It is obvious that whenever T is non-negative definite we have aj^(o;)raM(w) > 0 for any to. Thus a^(w)raM(ij) is a valid power spectrum. Furthermore, Theorem 6.4.1 indicates that in such a case l/aj^(u;)raA/(cj) is in fact the power spectrum of an (M l)th-order AR process. Consequently, Capon-1 yields an equivalent AR(M 1) process (whereas Capon-2 yields an equivalent ARMA(M 1, M 1) process). By making use of (6.12), we can find the coefficients of the equivalent AR process. The calculation of the exact covariance sequences from the AR coefficients is a standard problem and can be solved, for example, via the inverse Levinson-Durbin algorithm (See [75] [41] and the references therein for more details). Hence the implementation of the Capon method for covariance estimation runs as outlined below: Step 1: Pick up a value for M{M < N/2) and compute R by (3.13). Step 2: Compute ns associated with T = Rr1 by (6.12). Factorize Ml m=(M1) (say, by using the Newton-Raphson algorithm) and obtain the (minimum-phase) spectral factor. Step 3: Compute the corresponding covariance sequence (ftemp(^)} from the spectral factor (or, equivalently, the AR model) by, for example, the inverse Levinson-Durbin algorithm. While using the Fourier inverting method for covariance estimation, it is nec essary that the integral of the PSD estimate over all frequencies gives a good estimate of the signal power; otherwise there may be scaling errors in the covariance estimates. 75 or MAFI1, but at a significantly increased computational cost. Due to this observa tion, we do not recommend using this approach, i.e., minimizing (5.82), for refined estimation accuracy. Other more sophisticated techniques for system identification (see, e.g., [68] [54]) may be preferred in that event. Figure 5.5(c) also shows that, as compared to OEM, there is little computational advantage of using the initial estimates obtained by APES1 and MAFI1. The reason may be that the system in this example is quite simple (it has white output errors, etc.) and, apparently, OEM reaches convergence in a relatively small number of iterations. For a more complex system, such as the one used in the next example, OEM may need more iterations to converge. It should be mentioned that we did not program our method very care fully and hence our code is unlikely to be as efficient as the OEM code in MATLAB. Regarding the estimation accuracy, we shall stress that in the current case where the noise v{n) is white, OEM coincides with the optimal Maximum Likelihood Method (MLM) [68] [54]. When v(n) is colored, OEM is no longer MLM. In that case, the initial system parameter estimates obtained by APES1 or MAFI1 may outperform those by OEM, as in fact shown in the next example. Recall that LSEK is statistically efficient when the observation noise is white. Then, one might wonder why the initial estimates given by LSEK may be notably worse in such a case than those given by APES1 or MAFI1, as happened in the previous example (especially when N is small). The reason is that the transient response of this system cannot be neglected for small N. To show this, the PSD of x(n) is estimated by using the Capon PSD estimator, with N = 200 and M = 20, and is plotted in Figure 5.6. It shows two extra peaks (which behave like two sinusoids) at 0.1. The extra peaks are attributed to the response of the system (which has poles at 0.99e:D27r0'1) to the initial conditions. Since it is essential for LSEK to have the accurate knowledge of the number and frequencies of the sinusoids frequencies to give reliable amplitude estimates, its performance in the previous example is considerably 104 inverting the Capon PSD estimates in both the exact and approximate manners. The approximate Capon covariance estimates are computationally more convenient to obtain than the exact Capon covariance estimates, and the accuracy loss is in general very small provided that sufficient large frequency samples are used. It was found that the Capon covariance sequence estimator gives more accurate covariance estimates than the widely-used sample covariance sequences, especially for the higher lags. In summary, we have introduced the MAFI idea for spectral analysis and parameter estimation. shown that the Capon and APES complex estimators are both members of the MAFI approach to complex spectral estimation. proved that the estimation errors of the Capon and APES complex spectral estimators have similar asymptotic distribution. proved that for finite length of data samples, the Capon complex spectral esti mator is biased (downward) whereas the APES complex spectral estimator is unbiased, and that the bias of the forward-backward Capon is one half that of the forward-only Capon (to within a second-order approximation). devised efficient implementation schemes for the Capon and APES complex spectral estimators. presented a generalized MAFI approach, along with several other interesting methods, for amplitude estimation. demonstrated that the MAFI approach to amplitude estimation includes the WLS approach. CHAPTER 2 LITERATURE SURVEY Historical and modern perspectives on the general topics of spectral analysis and parameter estimation have been well documented in the literature [42] [62], Many classical articles, both theoretical and application-oriented, have been reprinted [20] [43]. Excellent texts are also available [75] [41] [55]. In this chapter, we give a brief review of a number of subjects that are related to our work, namely filterbank ap proaches to spectral estimation, the Capon method and applications, and covariance estimation. 2.1 Filterbank Approaches and Capon Estimator Unless the observed signal can be modeled with a finite number of parame ters, estimating the spectrum of a signal based on a finite length of observations is an ill-posed problem from a statistical standpoint, since we are required to estimate an infinite number of independent spectral values based on a finite number of sam ples. An assumption made by filterbank approaches and most other non-parametric methods is to assume that the PSD of the observed signal is (nearly) constant over a narrowband around any given frequency. Naturally one can proceed by passing the observed signal through a bandpass filter, which is swept through the frequency band of interest, and estimating the complex amplitude if complex spectral estima tion is of interest (or measuring the filter output power and dividing it by the filter bandwidth if PSD estimation is the desired goal), a procedure adopted by all filter- bank approaches. Obviously, how to choose the narrowband filters is a critical issue of filterbank approaches. Even though some of the classical Fourier-based methods, 9 72 that is, 1 i_1 Rvv = 7^ v(/)vH(/). (5.81) ^ 1=0 Step 2: Obtain estimates of {a,5,} by minimizing K 1 C2(a,b) = ^^-- \ak afc(a,b)|2. (5.82) *=i nuk) To do so we can use a host of methods, provided that we have good initial estimates of a and b. To obtain such estimates and then minimize (5.82), we assume that p and q are known. (Standard techniques for system order determination can be found in, e.g., [68] [54].) We pick up the p + q largest {d*} (if the SNR is low, an alternative is to choose those {a*,} that have the largest ratio |djt|2/^(wfe), assuming that was estimated) and define a criterion made from the corresponding terms of (5.82) P+Q , ^3(a,b) = ^^-- \ak afc(a,b)|2, (5.83) jfc=i amplitudes. Now, the minimization of (5.83) is simple. Indeed, almost always one can choose a and b to satisfy d* = afc(a,b), A: = 1, 2,... ,p + q. (5.84) Equation (5.84) is equivalent to T(e^) = R(e^fc), k = 1,2,... ,p + q, (5.85) 7 k which can be rewritten as a linear system of p + q equations with p + q unknowns {cii,bj}. That system will generally have a unique solution that makes (5.83) equal to zero, and which therefore gives our initial estimates of {a,6j}. As shown in the following numerical examples, the initial estimates are usu ally quite good. Hence, one can even skip the step of minimizing (5.82) to save computations. 55 Alternatively, we can rewrite (5.26) as y (0 = A iOl + e(/), where A, = A pjwii 0 pJUKl = AD,. We will use (5.26) mostly for analysis and (5.27) for estimation. The WLS (Markov-like) estimate of a in (5.27) is given by a = L-1 i -1 r L/=o L-1 Â£AfQ-'y(0 where Q is an estimate of Q = Â£{e(0e"(/)}. To estimate Q, we may proceed as follows. Let R=}Ey(0yffW. 1=0 One can verify that as L > oo, R goes to R = APA" + Q, where Hence, one way to estimate Q is as Q = R APA", (5.27) (5.28) (5.29) (5.30) (5.31) (5.32) (5.33) (5.34) where P is made from some initial estimates of {ak}k=\ obtained for instance via one of the LS amplitude estimators. The need for initial amplitude estimates is a BIOGRAPHICAL SKETCH Hongbin Li was born on March 30, 1970. He received the B.S. and M.S. degrees in electrical engineering from the University of Electronic Science and Technology of China (USETC) in July 1991 and April 1994, respectively. He was affiliated as an Assistant Lecturer at the UESTC from 1994 to 1995. Since 1996, he has been working toward his Ph.D. degree in electrical engineering at the University of Florida. 122 reduce the computational requirement are presented for the Capon and APES esti mators. For amplitude estimation, we describe a large number of estimators which can be categorized as the Least Squares (LS), Weighted Least Squares (WLS), and MAFI methods. While all these methods are asymptotically statistically efficient, their performances in finite length of data samples are quite different. Specifically, we show that the WLS and MAFI methods outperform the LS methods whenever the observation noise is colored; we also show that the MAFI approach is very general and includes the WLS approach as a special case. For covariance sequence estimation, we present a Capon covariance estimator by Fourier inverting the Capon Power Spectral Density (PSD) estimates. We describe the Fourier inversion in both an exact and approximate ways, of which the latter is computationally more attractive but with some minor accuracy loss. We show that the Capon covariance sequence estimates are in general better than the widely used sample covariance sequences. xii CHAPTER 1 INTRODUCTION This dissertation is concerned with spectral analysis and parameter estimation using MAtched-FIlterbank (MAFI) approaches. This chapter serves as a general introduction to the background and scope of the work. Significance and contributions are also highlighted in this chapter. 1.1 Background and Scope of the Work 1.1.1 Capon. APES, and MAFI Spectral Estimators Spectral estimation is an important data analysis tool which has found applica tions in many diverse fields including speech analysis, telecommunications, radar and sonar systems, biomedical and seismic signal processing, and economics [75]. There are two broad classes of approaches to spectral analysis: non-parametric approaches and parametric approaches. While non-parametric methods typically postulate no model for the observed data, parametric approaches do assume some model so that the spectrum is represented by a set of parameters, thereby reducing the spectral estimation problem to that of estimating the parameters of the model. Parametric methods may offer more accurate spectral estimates than non-parametric methods, provided that the data indeed observe the model assumed by the former. In the more likely cases where an accurate data model is not available, parametric methods will be sensitive to model mismatch; hence using the robust non-parametric approaches in such cases may be a better choice. Recent studies in the literature show that there has been renewed interest in non-parametric approaches to spectral estimation. 1 CHAPTER 6 CAPON ESTIMATION OF COVARIANCE SEQUENCES 6.1 Introduction Covariance sequence estimation is a ubiquitous task in digital signal process ing. A standard technique for estimating the covariance sequences is the so-called the standard sample covariance estimator. The standard covariance estimates are consistent provided that the given signals are ergodic to the second order. However, there is a major concern of using the standard estimator due to the unrealistic win dowing of the observed data it assumes; that is, it assumes that the data beyond the observed duration either is zero or repeats itself periodically. Partly for this reason, there have been several attempts in the literature to derive more accurate covariance estimates than the standard ones. A notable example is the approach based on the Burg autoregressive (AR) spectral estimator. However, the so-obtained covariance estimator was found even less accurate than the standard sample covariance estima tor [78]. To be more exact, the Burg approach was shown to have larger variances than the standard method. Another approach, which has generated a whole new research direction, relies on the maximum likelihood (ML) principle [13]. However, the ML estimation of covariance sequences is a computationally involved problem which does not have a closed-form solution. The solution given in [13] is iterative and not guaranteed to be globally optimum. Apparently, there exist no compelling alternatives to the standard method that can be recommended for general use. In this chapter we present a new method, namely the Capon method [48], for covariance sequence estimation. The Capon method obtains the covariance sequence estimates by Fourier inverting the Capon power spectral density (PSD) estimates. 80 APPENDIX B PROOF OF THEOREM 3.3.1 (We give below a proof of Theorem 3.3.1 only for the FB case. The proof for the FO case follows a similar (and simpler) pattern.) From (3.4) and (3.7), we have g = aaM + <5, g = &&m + , where and S, respectively, are defined as S = ^ 1=0 i = 4xs(0e_i"'- ^ 1=0 First we calculate the first and second-order moments of <5 and : E{6} E{~6} Â£<[<5 = 0, = 0, = 0, = 0, ;=o fc=o L-i = J X ( ||)R.(<)e-^, i=-(L-l) 1=0 k=0 L-i = 5 X ( |i|)R.(i)e->*. fe-(L-l) (B.l) (B.2) (B.3) (B-4) (B.5) (B.6) (B.7) (B-3) (B.9) (B. 10) 108 xml version 1.0 encoding UTF-8 REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd INGEST IEID EDPQUYX16_3P9GPE INGEST_TIME 2014-12-08T21:52:33Z PACKAGE AA00024495_00001 AGREEMENT_INFO ACCOUNT UF PROJECT UFDC FILES 54 Note that, for most cases of interest, LSE(1,0,A') will give more accurate amplitude estimates than LSE(1,0,1), and that the difference between these two estimators is small for large N. On the other hand, LSE(1,0,1) is computationally more efficient than LSE(1,0, K) since the matrix multiplication and inversion in (5.4) are avoided. Hence LSE(1,0,1) may still be worth considering. 5.3 WLS Amplitude Estimators If we split the data vector x into subvectors, then the covariance matrix of the noise part of the subvectors may be estimated and can hence be used to derive an optimal WLS estimator (i.e., a Markov-like estimator) [68]. In this section, we describe a number of such WLS estimators that split the data into vectors of shorter length, utilize no prefiltering, and estimate either one or K amplitudes at a time. 5.3.1 WLSEfL, 0. K) We define the following subvectors y (0 = where 1 T x(l) x (l + 1) ... x{l + M 1) L = N -M + 1. I = 0,1,..., L- 1, (5.23) (5.24) The choice of M (M can be chosen smaller than K. See Figure 5.4. Moreover, when M 1, all WLSE(L, 0, K) reduce to LSE(1, 0, K)) or, equivalently, of L is discussed in Section 5.5. We have , (5.25) i 1 axe^1 v(l) y(0 = ejwi eJ0JK + v(l + 1) ej(M-l)un ej{M-l )ujk apce^1*1 v(l + M 1) or, with obvious notation, y(0 = As(0 + (0- (5.26) Figure 6.4: True covariance sequences, (a) ARMA1; (b) ARMA2; (c) ARMA3; Covariance Sequences p f J l l l Covariance Sequences Q- 3 ? 01 (Q A _ O 1 ARMA3 ARMA4 Covariance Sequences CD Cn 44 For similar reasons as for APES, the intuitive implementation of Capon given in (3.52) is computationally more involved than the efficient implementation of Capon proposed above. 4.4 Numerical and Experimental Examples We present numerical and experimental examples comparing the performances of APES and Capon with the FFT methods [52] for SAR imaging. In the following examples, we choose M = N/2 and M = /2 for both Capon and APES. For the windowed FFT method, we use the Kaiser window with parameter 4. We first consider SAR imaging of a simulated MIG-25 airplane. The 32 x 32 data matrix was provided by the Naval Research Laboratory. The 128 x 128 SAR image obtained by using 2-D FFT, 2-D windowed FFT, 2-D Capon, and 2-D APES are shown in Figures 1.1(a) to 1.1(d), respectively. We note that Capon and APES outperform the FFT methods. The number of flops required by our efficient ways of implementing Capon and APES are about 950 and 1500 times of those required by the FFT methods, while those required by the intuitive ways of implementing Capon and APES are about 22800 and 30000 times, respectively, of those required by the FFT methods. That is, the number of flops required by the intuitive ways of implementing Capon and APES are, respectively, about 24 and 20 times of those required by our efficient ways of implementing them. If we increase the size of the image to 256 x 256 and 512x512, respectively, the ratios of the needed flops between the intuitive ways and our new ways for implementations are 36 & 40 for Capon, and 31 & 34 for APES, respectively. We now consider an example of SAR imaging with experimental data. The data matrix is 64 x 64 and is obtained from the experimental data collected by one of the two apertures of the ERIMs (Environmental Research Institute of Michigans) DCS IFSAR (interferometric SAR). The 256 x 256 SAR image obtained by using 2-D 19 3.2 MAFI Filters By definition, the matched filter is designed such that the corresponding signal-to-noise (SNR) ratio in the filter output is maximized; that is, \h%aM(u)\2 arg max tr h"QMhw (316) The solution is obtained by making use of the Cauchy-Schwartz inequality (see, e.g., [75]): _ Q-a mM , " aÂ£(u>)Q-i(u)a(w) 10 where Q(u>) is assumed to be invertible. It is readily checked that the solution in (3.17) satisfies h"aMM = l, (3.18) which implies that the filter given in (3.17) passes the frequency uo undistorted. By making use of this observation and of (3.4) and (3.7), we have h5y(Z)=a(w)e'wi + h^iw(0, / = 0,1,..., L 1, (3.19) and h?y(0 = e-*lf-1*a'{u>)eul + l = 0,1,..., L 1. (3.20) The least squares (LS) estimate of (a;) obtained by using only (3.19), i.e., the forward data vectors, is given by FoM = (3-21) whereas the least squares (LS) estimate of a(u) obtained by using (3.19) and (3.20), that is, both the forward and backward data vectors, is given by fbM = ^ [h^fg(w) + e~N-1)uJgH(u)K] (3.22) where g(w) and g(w) are, respectively, the normalized Fourier transforms of the forward and backward data vectors: iM 1 L 1=0 (3.23) 40 and Let L\ 1 Z/2 1 V>(wi,w2) = ^2y{h,l2)e~UJlll+U2l2). h= 0 2=0 (4.4) Z = y(0,0) y{L\ 1,0) y(0,L2 1) y(Li 1,Z,2 1) (4.5) and Z = JZ*J, where J denotes the exchange matrix. We can then rewrite (4.3) and (4.4) as ^(wi,W2) = ZaliL2(o>i,u;2), (4.6) and ^(wi,w2) = Za12 (wi,o;2). (4.7) By applying the matrix inversion lemma, we obtain Q-(a)l,^) = Q"K,^)+^1("1-a,!)y"1U,2)j,"(u,1'U,2)^1(a1'^), (4.8) LiL2-V> (wi,w2)Q-1(a;i,a;2)'0(cj1,a;2) where Q-1(a,^)=R-1+ (49) LL ip (a;i, a;2)R_1V> (a>i, cj2) (For notational convenience, we sometimes drop the dependence on cji and cj2 below.) Hence, and aMx,M2(w iwi)Q 1(c^i, o;2)t/j(c^i, a^2) k-i *mumA lWHQ > ,Q_1^ + aMuM2 LiL2&mum2Q 1^ LiL2 tpHQ-1xp L\L2 ipnQ-'-'ip (4.10) aMi,M2(Wliu;2)Q 'aMi.Mj^l,^) lMi,M2 Q 'aMi,M2 + ,M2 Q-1^ H, LiLi-ip Q_1i/> (4.11) 27 APES estimators are related by LE {FBC(u) a(uj)} LE{aF0C(u) a(w)} /0 7 \ = 0 7 T < 0, (3.57) a[io) 2 Oi(ij) and LE {AFbaM MI = EE {foa(w) ~ a(w)} = 0, (3.58) for sufficiently large values of L. Proof: See Appendix C. We believe that (3.57) and (3.58) provide a theoretical motivation for prefer ring APES to Capon in most spectral estimation exercises. Moreover, Theorem 2 also suggests that FBC should be preferred over FOC. While both FOA and FBA are similarly unbiased (within a second-order approximation), the latter is usually observed with slightly better resolution and sidelobe properties [52] at the cost of slightly more computations. 3.4 2-D Extensions We briefly describe the 2-D extensions of the MAFI spectral estimators. We first decompose the observations {y(ni,n2)} as t/(ni,n2) = o:(a;i,W2)e-j(Jini+a;2n2)+eWliW2(n1,n2), (3.59) nx = 0,1,..., W 1; n2 = 0,1,...,N2 1; uq, cj2 quency (u>i,u>2) and eUliU2(ni, n2) denotes the noise (or residual) term at frequency (uq,u;2), assumed to be zero-mean. Next, in a manner similar to the 1-D case, we form the Mi x M2 forward and backward data matrices: Y(u2) = {y(n\,n2), n\ = Zi,..., l\ + M\ 1; n2 Z2,..., l2 + M2 1} , Y(Zi,Z2) = {y*(ni,n2), nx = Ni-li-l,...,Ni-li-Mi] n2 = N2 l2 1,..., N2 l2 M2} , h = 0,1,..., Li 1; l2 = 0,1,..., L2 1, (3.60) LIST OF FIGURES 1.1 Synthetic Aperture Radar (SAR) images of a simulated MIG-25 air plane obtained by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES 3 3.1 The 1-D complex amplitude of the sum of 15 sinusoids used in the simulations, (a) Real part; (b) Imaginary part 33 3.2 Empirical bias and variance of the 1-D MAFI estimators as SNRi varies when IV = 64 and M = 15. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the esti mated amplitude; (d) Variance of the imaginary part of the estimated amplitude 34 3.3 Empirical bias and variance of the 1-D MAFI estimators as the filter length, M, varies when N = 64 and SNRi = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude 35 3.4 Empirical bias and variance of the 2-D MAFI estimators as the SNRi varies when Nx = N2 = 32 and Mx = M2 = 8. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude 37 59 where (-)1/2 denote the Hermitian square root of the positive definite matrix argu ment. Observe that X is semi-unitary, i.e., XHX = lR. (5.48) The cost function in (5.46) can now be rewritten as / = tr x"q-1/2apahq-1/2x] . (5.49) It follows from the Poincar separation theorem (or the generalized Rayleigh quotient theorem) [37] that R max / = Xk (CT1/2APA^Q-1/2) (5.50) k=1 where {Afc(-)}j^.1 denote the eigenvalues of the matrix between the parentheses, or dered such that Ai > A2 > ... > furthermore, the columns of the maximizing X are equal to the eigenvectors corresponding to {Afc}^=1. Next, note that post-multiplying X by any unitary matrix of appropriate dimensions yields another valid solution for X. One such solution having a simple form can be obtained as follows. Observe that rank (q1/2APAhQ-1/2) = K, (5.51) which implies that we cannot improve the generalized SNR by choosing K > K since = ... = XR = 0. On the other hand, the larger the K the more filtered data will be available for amplitude estimation. Hence, we choose K = K. In such a case, the maximizing X is given by (5.52) X = Q"1/2AT (5.53) 51 Next, since (see, e.g., [34]) JSo^(") = I"' <5-8> where 1^ denotes the N x N identity matrix, and lim 4("W) = 7V->oo N 0 <>{wk) the asymptotic MSE is given by (5.9) limIVMSE{a} = NÂ¥00 0 <Â¡>{uk) (5.10) Under the mild assumption that v(n) is circularly symmetric Gaussian, the CRB for a is given by (see, e.g., [68]) CRB{a} = (iW_1)_1. (5.11) Using the following result (see [34] once again) lim 4(HW_1) = N-+oo N (/) L(W!) (j) 1 (to k) (5.12) we obtain lim 7VCRB{tx} = N>oo 0 0(wk) (5.13) which coincides with (5.10). Remark: It can be readily checked from (5.6) and (5.11) that if v(n) is white, i.e. W ~ 1N, then LSE(1,0,R) is statistically efficient for all N > K. 53 where a = U ... CtK ]T and A is defined through A (5.18) Hence, LSE(1, 0,1) is biased. However, it is asymptotically unbiased (that is, its bias goes to zero as N oo). We next calculate the MSE of on: MSE{i} = (H) 1aH [Acta11 AH + W) (H) 1. (5.19) Making use of (5.9) once again, along with the fact that AHa/\f 0 as N > oo, we have lim AMSE{d;i} = (Â¡>{l\). (5.20) N-+oo Hence, LSE(1,0,1) is also asymptotically efficient. On the other hand, in finite samples (5.14) may be better or worse than (5.4), depending on the characteristics of the scenario under study. The fact that (5.4) may be better than (5.14) comes as no surprise. As an example, let us assume that the Signal-to-Noise Ratio (SNR) is high. Then, the bias of (5.14) dominates the variance part. On the other hand, (5.4) has no bias and its variance will be smaller than the bias of (5.14) if the SNR is large enough. Consequently, the MSE of (5.4) will be smaller than that of (5.14). The fact that (5.14) may be better than (5.4) is however a surprise. For an example of such a case, assume SNR (5.21) whereas for (5.4), MSE{d!} (A^A)-1 i,i (5.22) which can be much larger than (5.21) (e.g., if \u>k Wi| ~ l/N for some k > 2). In (5.22), [-jij- denotes the ij-th. element of the matrix argument. 105 presented a new covariance sequence estimator by Fourier inverting the Capon PSD estimates. described how to Fourier inverting the Capon spectra exactly, which is an in teresting endeavor by itself. 7.2 Future Work One interesting area of future work is amplitude estimation for 2-D sinusoidal signals (with application to 2-D system identification). However, unlike the extension of the MAFI complex spectral estimators from the 1-D case to the 2-D case, the extension for amplitude estimation is not so straightforward. A first question would be the analysis of LSE(1,0, K) and LSE(1, 0,1) in the colored noise case. There may be no compact expression for the asymptotic accuracy that is similar to the one in the 1-D case. The reason is that there exist in the 2-D case no such expressions as those shown in (5.8) and (5.9). Another important question would be how to apply WLS to the 2-D data. This involves how to create the 2-D subvectors from the data matrix, and how to estimate the covariance matrix of the noise subvectors, etc. One can expect that the 2-D WLS amplitude estimators may be considerably more complex than their 1-D counterparts. Yet, as in the 1-D case, the 2-D WLS (or MAFI) amplitude estimates should in general be more accurate than the simple LS amplitude estimates. 121 [80] H. Wang and L. Cai. On adaptive spatial-temporal processing for airborne surveillance radar systems. IEEE Transactions on Aerospace and Electronic Systems, 30(3):660-670, July 1994. [81] H. Wang and G. H. Wakefield. Signal-subspace approximation for line spectrum estimation. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pages 2054-2057, Dallas, TX, April 1987. [82] J. Ward. Space-time adaptive processing for airborne radar. Technical Report 1015, Lincoln Laboratory, MIT, December 1994. [83] B. Widrow, P. E. Mantey, L. J. Griffiths, and B. B. Goode. Adaptive antenna systems. Proceedings of the IEEE, 55(12) :21432159, December 1967. [84] D. M. Wilkes and M. H. Hayes. Iterated Toeplitz approximation of covariance matrices. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pages 1663-1666, New York, April 1988. [85] D. B. Williams and D. H. Johnson. Robust estimation of structured covariance matrices. IEEE Transactions on Signal Processing, 41(9):2891-2906, September 1993. 63 Figure 5.1: PSD of the test data that consist of three sinusoids and an AR(1) noise process. where fc(z) is the estimate of a.k derived in the zth simulation run. 5.5.1 Estimation Performance versus SNR First, we consider the case where v(n) is colored. More exactly, v(n) is de scribed by the following AutoRegressive (AR) process v(n) = 0.99u(n 1) + e(n), (5.69) with e(n) being a complex white Gaussian noise with zero-mean and variance a2. The PSD of the test data is shown in Figure 5.1, where a2 0.01. The local SNR of the k-th. sinusoid is defined as [41] Iry, N\cu P SNRfc = 10 log10 10 logio -rpr. (5.70) / Hf)df nJk> Note the occurrence of N in the above SNR formula. For those methods that depend on M, we choose M = N/4 = 8, giving L = 25 (see Section 5.5.2 for a study of the effect of M on the performance). Figure 5.2(a) shows the MSEs of the seven amplitude estimators for 0:3, along with the corresponding CRB, as the SNR varies. As one can see, APES1, APESK, and MAFI1 are very close to the CRB, while LSEK, which ignores the noise cor relation, is evidently away from the CRB. CaponK also deviates from the CRB for 82 better AR coefficient estimates are obtained by using the Capon covariance esti mates than by the standard ones. Another application discussed in this chapter is the moving-average (MA) model order determination by making use of the Capon and the sample covariance estimates, where we find that better performance is achieved by the former. It should be mentioned that we can obtain APES covariance sequence esti mates similarly by Fourier inverting the APES PSD estimates. However, in spite of the evident difference of the Capon and APES complex spectral estimates for discrete spectral components, their performances in continuous PSD estimation are similar to one another, which implies that the APES covariance estimates are similar to the Capon covariance estimates. As such the APES covariance estimation is not discussed herein. 6.2 Standard Covariance Estimator and Outlook With no other assumptions made on the signal under study, except for assum ing the second order ergodicity, there are two ways to obtain the standard sample covariances of the signal, namely, the biased and the unbiased covariance estimators. However, the biased covariance estimator is more commonly used since it provides smaller mean-squared errors (MSE) than the unbiased one and guarantees the co- variance estimates to be positive semidefinite [75]. The biased sample covariance estimator of a wide-sense stationary signal with zero-mean has the form JV-fc (6.1) k = 0,1,..., K. 72=1 where {y(n)}%=1 are the observed data samples, N is the number of samples, f(k) denotes the estimate of the covariance function r(k), K is the largest lag desired (0 < K < N 1), and ()* denotes the complex conjugate. Note that (6.1) is asymptotically unbiased. 89 So after Step 3, we use a power compensation approach to obtain our ultimate Capon covariance estimates. Our final estimates {^Caponi^)} have the form n \ f) ^temp(^) rCPnWPV,e1p(0) where P0 is the estimated power of the signal n=0 (6.13) (6.14) 6.4.2 Approximate Method Since the Capon-1 spectrum is equivalent to an AR spectrum, the covariance sequence can be computed exactly as described above. With some accuracy loss, the computational demand of the Capon method can be reduced. According to the Wiener-Khintchine theorem, we have r(k) (6.15) We can rewrite (6.15) as 1 r2n r(k) / (t){u)elkudu. (6.16) Jo Let 3> N, and let the Capon spectrum be evaluated at u> = 2ir/\ that is, we calculate (o>), = 0,..., 1. Then we can approximate the computation of the covariance sequence corresponding to 4>(uj) by the following equation f(k) 1 2tt 27t IV Nl J2 1 N N-1 J2 j>(un)eikU, =0 (6.17) which can be evaluated by using FFT. Since the error in approximating the integra tion by the summation above is 0(l/iV), the errors introduced are quite small for large enough . Our computer simulations also confirm this observation. The evaluation of (f)(uj) in (6.17) by directly using (6.6) is computationally burdensome. We can instead make use of (6.11) as follows. Since R_1 is Hermitian, 85 sample covariance matrix, given in (3.13), for better statistical properties associates with the latter [41]. Another more elaborate choice of (3 is obtained as the equivalent bandwidth of \H(u>)\2, where H(u>) is the filters frequency response: l h H(w) = [h^"] a M(w). (6.7) This specific bandwidth choice leads to the Capon-2 PSD estimator [45] [75]: a&(w)R-1aM(w) Capon-2: = (6.8) a^(o>)R-2a M{u) Burg showed that the inverse of Capon-1 spectrum is equal to the average of the inverses of the estimated AR spectra of orders from 0 to M [11]. This observation reveals the fact that Capon-1 has less statistical variation as well as lower spectral resolution than the AR estimator. A similar but more involved relationship between Capon-2 and the AR estimators was derived in [75]. Theoretically, the performance of Capon-2 is hard to quantify. However, it is generally believed that Capon-2 possesses finer resolution and hence is a better spectral estimator than Capon-1 [45]. We will show here, with a typical example, that even though Capon-2 has better resolution locally around the power peaks, it is globally a more biased estimator than Capon- 1. Our experience also shows that Capon-2 generally gives much poorer covariance estimates than Capon-1. Therefore, Capon-2 is not recommended for covariance sequence estimation. To illustrate the above claim, consider an ARMA(4,2) signal y(n) = 2.76y(n 1) 3.809y(n 2) + 2.654y(n 3) 0.924y(n 4) +e(n) 0.9e(n 1) + 0.81e(n 2), (6.9) where e(n) is a real white Gaussian random process with zero-mean and unit variance. The Capon-1 and Capon-2 spectral estimates with N = 256 and M = 50 are shown in Figure 6.1(a), where the dashed curve stands for the true PSD of the ARMA 32 almost unbiased, while FOC and FBC are biased downward. In addition, we notice that the bias for FOC is approximately twice that of FBC. All these observations are consistent with the prediction of the theory. The variances of the real and imaginary parts of the amplitude estimates are shown in Figures 3.2(c) and 3.2(d), respectively. It appears that all of the estimators display similar variances. However, as shown in the next example, the variance of Capon becomes notably larger than that of APES as M increases. Next we study the effect of the filter length, M, on the estimators. The SNRi is fixed at 20 dB. As M varies, the real and imaginary parts of the bias are shown in Figures 3.3(a) and 3.3(b), respectively. From these figures, one can see that both FOA and FBA are unbiased for all practical filter lengths, whereas the bias of Capon grows significantly with increasing M. (A practical filter length means that M should not be too small [52]. In fact, all filterbank methods reduce to the Fourier transform approach when M 1, and only when M is sufficiently large, the filterbank approach shows noticeable improvement over the Fourier method [52].) All estimators seem to perform similarly for M up to a fourth of the data length, with Capon being slightly biased downward. As the filter length increases further, the performance of Capon degrades rapidly, while that of APES remains unaffected. This observation is strengthened by the variance results shown for the real and imaginary parts of the amplitude estimates in Figures 3.3(c) and 3.3(d), respectively. It is known that, as M increases, all of the estimators under study achieve better spectral resolution and that the best resolution is obtained at M = N/2 [52]. This fact, along with the statistical results shown in the previous examples, indicates that the choice of M for Capon should be made by a tradeoff between resolution and statistical stability. Usually we choose N/4 < M < N/2. While the choice of M for Capon is difficult to make, it is easy to see that APES achieves the best performance at M = N/2, since with this choice, APES achieves the highest possible CHAPTER 3 MAFI APPROACH TO SPECTRAL ESTIMATION Filterbank approaches decompose the observations {y(n)}^=01 of a stationary signal y(t) as [51] [47] [73] [72] y{n) o(u)ejun + ew(n), n = 0,1,..., N 1; wG[0,27r), (3.1) where a(co) denotes the complex amplitude of the sinusoidal signal with frequency u> and ew(n) denotes the noise (or residual) term at frequency u>, assumed to be zero-mean. The problem of interest is to estimate a(u) for any given u>. Briefly stated, most filterbank spectral approaches address the aforementioned problem by following two main steps: (a) pass the data {y(n)} through a bandpass filter with varying center frequency u; and (b) obtain the estimates, (w), for u G [0, 27t), of the complex amplitude from the filtered data. The bandpass filter used is usually an M-tap FIR filter with its coefficient vector given by r -\T hw = h\ h 2 h M (3.2) where (-)T denotes the transpose. (The choice of M is discussed in Section 3.5.) Observe that the notation emphasizes the dependence of the vector in (3.2) on the center frequency lo. Although rules for choosing h^, vary, a rather general one for the choice of a matched filter is discussed in Section 3.2. 3.1 Forward and Backward Data Vectors Let y (0 = y(0 y( + !) y(l + M -1) Z = 0,1,..., L 1, (3.3) 16 25 and ug(uj)v2(u) fbaM II^iHII2 \ K(w)^H 1/f (w)i/3(w)] S ^w) (3.53) where (3.54) Hence computationally APES, especially FOA, is only slightly more involved than Capon. (Also see Section 3.5 for the simulation results.) More specifically, the amount of computations required by Capon or APES is dominated by calculating R 1//2 and the matrix-vector products in (3.44)-(3.45) or (3.49)-(3.51). We mention that conventional Capon or APES implementation makes use of ((3.46), (3.47)), ((3.52), or (3.53)), which requires calculating the matrix-vector products in (3.44)- (3.45) or (3.49)-(3.51) for each w of interest, thereby becoming computationally more and more intensive as the number of frequency samples increases. This is especially so in 2-D applications such as when forming SAR images. It is thus of great interest if other efficient implementation schemes for the MAFI approaches can be found. 3.3.2 Statistical Performance All estimators under study, i.e., FOC, FOA, FBC, and FBA, can be shown to have the same asymptotic variance under the following condition: C: The signal y(n) can be written as in (3.1), where ew(n) is a zero-mean stationary random process with finite spectral density at u: 4>e{u) < OO. (3.55) In more exact terms, the following result holds true. 99 ARMA1 (N=256, M=33,100 realizations.) ARMA2 (N=256, M=33,100 realizations.) (a) (b) (c) (d) Figure 6.6: The AR coefficient estimation of the ARMA signals via the overdeter mined modified Yule-Walker method with N = 256 and M = 32. The curves are the summations of the mean-squared errors (MSE) of all the AR coefficient estimates versus the numbers of included equations, which have been set as 4, 8, 16, 32, 64 and 128, respectively. The MSE curves are based on 100 independent realizations, (a) ARMA1; (b) ARMA2; (c) ARM A3; (d) ARMA4. 90 by Theorem 6.4.1 we have Â¡is = Â¡i*_s. Then M1 Mf1^) = Y, ^eJSUJ s=(M1) M-1 0 E + E ft) M1 s=0 s=(M1) M-l Af-1 E ^ /_se-^* /io ,-JSiOfi s=0 ^(W) + 01 (k>fi) A0, (6.18) where 1V-1 (6.19) s=0 which, again, can be evaluated by using FFT with zero-padding, i.e., = 0, for s = M + 1,..., 1. 6.4.3 Computational Aspects We briefly discuss the computational aspects of the standard method and the Capon method for covariance sequence estimation. Recall that K, M, and N denote the largest lag of the covariance estimates, the length of the Capon filter, and the number of data samples, respectively. Assume that the data is real, N K, N M, and K M. Then the standard method involves approximately 2KN flops. The number of flops needed by the exact Capon method is difficult to deter mine. For a moderate M, say N/6 < M < N/3, we find that the computationally most demanding part of the exact Capon method is the spectral factorization. Since the algorithm (Newton-Raphson) used to find the minimum-phase spectral factor is iterative, the number of flops needed is hard to quantify. Another significant compu tational demand is the calculation of the sample covariance matrix R, which involves approximately 2NM2 + 4M3 flops. However, we note that the amount of computa tions needed by the factorization is much larger than that needed for computing the sample covariance matrix for a moderate or large M. 71 For sufficiently large N (so that the transient response in the output can be neglected), the cost function CoEM(a)b) is approximately equivalent to N-l Cd a,b) = Â£ n=0 K x(n) a,b)eJI fc=i (5.77) The method that we propose for estimating a and b is based on (5.77) and consists of two steps: First estimate {ak}kz=1 in an unstructured/non-parametric form. Then fit {(a, b)}^ to the amplitude estimates obtained in the previous step by taking into account the statistical variance of the latter. In what follows, we detail the above two steps. Step 1: Use an appropriate amplitude estimator to obtain estimates {k}k=1 of {oik}k=i from the measurements {^(n)}^1. APESl may be recommended in this case because we have control over the probing signal and we have no reason to choose any of the sinusoids too close to one another. The large-sample variance of the estimated amplitudes {&k}%=l is proportional to {(f)[oJk)}k=i (see Section 5.2). To obtain estimates of {(Â¡>{^k)}k=v we can first calculate K v(n) x(n) y] ke3Lkn, n = 0,1,..., N 1, (5.78) k=1 and then utilize either a parametric or a non-parametric PSD estimator [75] [41] on (5.78) to obtain {(uk)}k=1. Specifically, in the examples given in Section 5.6.2 we use the Capon PSD estimator [16] [44] [75] (also see Section 6.3), which determines { M k = l,2,...,K, (5.79) where a(uk) is defined in (5.40) and Rvv is the sample covariance matrix of the estimated noise vectors v(0 = v(l) v(l + 1) v(l + M 1) 1 = 0,1,...,L-1, (5.80) 57 From (5.41), we can estimate f3k one at a time via LS as 1 L_1 ft = 7Ey^e_Wi^g^ k = 1,2,..., K. (5.42) L 1=0 (Note that we could estimate all {/3k}k=i simultaneously via LS, which however appears to perform even worse than using (5.42), especially for small N.) The use of (5.42) in (5.32) and (5.39) leads to the following estimate of Q K Q = R- (5-43) k=1 The WLSE(L, 0, K) that uses (5.29) with (5.43) does not require any initial estimate of {ak}Â£=v It is an extension of the APES algorithm in [52] to multiple sinusoids with known frequencies. Remark: We note that e(k) and e(Z) in (5.27) are correlated (for k ^ l), which implies that (5.29) is suboptimal (as it takes into account only the correlation between the elements of e(Z), but ignores the correlation between e(l) and e(fc), for k ^ l). Yet, the WLS methods are likely to outperform the LS methods because the latter completely ignore the correlation in v(n). 5.3.2 WLSEL, 0.11 The particularization of WLSE(L, 0,K) to WLSE(L,0,1) is straightforward. Specifically, the WLSE(L, 0,1) that corresponds to using (5.29) with (5.34) can be readily verified (by using the matrix inversion lemma) to be a7i(wfc)R_1g(u;fc) Â£*Jfc A; = 1,2,..., AT, (5.44) af7(u;fc)R-1a(a;fc) whereas the WLSE(L, 0,1) that corresponds to using (5.29) with (5.43) is given by a H{u>k) R g(wfc)g/i(wA;) -l g(wjt) a"K) R g{iok)gH(cok) -l a(wfc) k = 1,2,..., K. (5.45) Note that (5.44), like (5.38), does not depend on P. However, unlike (5.38), the equation (5.44) is exactly equivalent to using (5.29) with (5.34). Equations (5.44) and 17 be the overlapping vectors constructed from the data {y(n)}, where L = N M+l. In what follows y(Z) is referred to as the forward data vector. Let ew(Z), l = 0,1,..., L1, be formed from {ew(n)} in the same manner as y(Z) are from {y(n)}. Then the forward vectors can be written as y(0 = WuNMje1 +e(0, (3.4) where is the steering vector and is given by a m(^) 1 T 1 eJU (3.5) Likewise, the backward data vectors are constructed as m y*(N-l- 1) y*(N-l- 2) ... y*{N-l-M) Z = 0,1,..., L 1, (3.6) where ()* denotes the complex conjugate. Let aJ(Z), Z = 0,1,..., L 1, be formed from {ew(n)} the same way as y(Z) from {y(n)}. Then the backward vector can be written as y(Z) = [tt(w)aM(w)]e,'wi + ea,(Z), (3.7) where (a;) = a*(a;)e_^JV1^. (3.8) It is straightforward to verify that the forward and backward vectors are related by the following complex conjugate symmetry property: y(z) = Jy*(T z l), (3.9) where J denotes the exchange matrix whose anti-diagonal elements are ones and all the others are zero. Suppose that the initial phase of the sinusoidal signal in (3.1) is a random variable uniformly distributed over the interval [0, 2tv) and independent of the noise 10 such as periodogram by Schuster [67] and its various variations including the famous Blackman-Tuckeys method [5], can also be cast in the framework of filterbank ap proaches, originally they were not designed in such a manner. In other words, those methods made no attempt to purposely design a good bandpass filter to achieve some desired characteristics. A notable example of filterbank approaches to spectral estimation is the RE- FIL (REfined FILter) method which was first introduced in [77] and was further developed in [56] (also discussed in [8] [59] [61]). The REFIL method is close in spirit to the Daniell approach [22] to reducing the variance of the periodogram. That is, REFIL does not split the available samples in shorter stretches. The REFIL idea is to design a bank of filters which pass the signal components within the passbands as much as possible relative the total power and, in the mean time, attenuate the frequencies outside the corresponding passbands. It turns out that REFIL in general gives better spectral estimates than the traditional Fourier-based methods. Neverthe less, the REFIL filters share a common characteristic with the Fourier-based ones: they are all data-independent in the sense that they do not adapt to the received data in any way. Presumably, it would be beneficial to take the data properties into account when designing filterbanks. The Capon method is one (and perhaps the most famous one) of such data- dependent filterbank spectral estimators. The Capon estimator was first named as the Maximum Likelihood Method (MLM) due to the the ML and Gaussian process context used in Capons original work [16]. It turns out that MLM is a misnomer since it is not an ML spectral estimator and it does not possess any of the properties of an ML estimator. In [41] and [55], Capons method was referred to as the Minimum Variance Spectral Estimator (MVSE) because it is derived by minimizing the variance of the output of a narrowband filter. Even the name MVSE is inaccurate in the sense 3.5Empirical bias and variance of the 2-D MAFI estimators as the filter length, M = Mi = M2, varies when Ni = TV2 = 32, and SNRx = 20 dB. (a) Real part of the bias; (b) Imaginary part of the bias; (c) Variance of the real part of the estimated amplitude; (d) Variance of the imaginary part of the estimated amplitude 38 4.1 SAR images obtained from the ERIM data by using (a) 2-D FFT, (b) 2-D windowed FFT, (c) 2-D Capon, and (d) 2-D APES 46 5.1 PSD of the test data that consist of three sinusoids and an AR(1) noise process 63 5.2 Empirical MSEs and the CRB versus local SNR when TV = 32, M = 8, and the observation noise is colored (an AR(1) process), (a) a3. (b) ax. 65 5.3 Empirical MSEs and the CRB versus local SNR when TV = 32, M = 8, and the observation noise is white, (a) a3. (b) ax 67 5.4 Empirical MSEs and the CRB versus M when TV = 32 and the obser vation noise is colored (an AR(1) process with o2 = 0.001). (a) o;3. (b) ax 69 5.5 Averaged RMSEs and the number of flops versus TV for the first system when the observation noise is white (o2 = 0.01) and M 20 for APES1 and MAFI1. (a) RMSE of a-parameters. (b) RMSE of 6-parameters. (c) Number of flops 76 5.6 PSD estimate of the output of the first system corrupted by white noise with a2 = 0.01 and TV = 200 77 5.7 Averaged RMSEs and the number of flops versus TV for the second system when the observation noise is colored (an AR(1) process with a2 = 0.01) and M = 20 for APESl and MAFI1. (a) RMSE of a- parameters. (b) RMSE of 6-parameters, (c) Number of flops 78 ix 18 term. By making use of this assumption as well as (3.9), the covariance matrix of y(l) or, equivalently, of y(l), is given by R = E {y(l)yH(l)} = E {y(Z)yH(0} = lM|2 aM(w)a^(w) + Q(w), (3.10) where (-)H denotes the conjugate transpose and Q(cu) is the noise covariance matrix and is given by QM = B{M0"(0} = B{Â§(0Â§5(0} (3.11) Note that both R and Q are Hermitian Toeplitz matrices. The Forward-Only (FO) interbank approaches use only the forward data vec tor to estimate R, which is the forward sample covariance matrix: 1 L-1 R-fo = y y(0yH(0- (3-12) ^ 1=0 The Forward-Backward (FB) filterbank approaches use the average of the forward and backward sample covariance matrices to obtain the estimate of R: R-fb = ^(Rfo + Rbo)i (3-13) where R-bo denotes the backward sample covariance matrix: 1 L_1 RBO = 7E3'Wy"(,)=jftFOJ- (3-!4) ^ 1=0 The Rfb in (3.13) is Hermitian but no longer Toeplitz. By using (3.9), one can show that Rfb is a persymmetric matrix [30], i.e., RFb = JRfbj- (3.15) Since R is persymmetric, one would expect that R-fb is a better estimate of R than the non-persymmetric Rpo- 2 Filterbank approaches to spectral estimation belong to the class of nonpara- metric spectral estimators. An important member of filterbank approaches is the Capon spectral estimator [16] [18]. Unlike the classical Fourier-based methods which are data-independent, the Capon spectral estimator adapts to the processed data in a manner so that the noise components of the data are rejected substantially. During the past few decades, the Capon spectral estimator has been widely used because its higher resolution and lower sidelobes give it an advantage over the Fourier-based methods. Additionally, its robustness and less variability make it preferable to the parametric methods [75] [41]. In a recent study, it was empirically observed that using the Capon estimator for complex spectral estimation gives biased spectral estimates whereas the newly introduced APES (Amplitude and Phase Estimation) method appears to be unbiased [52] (see Figures 1.1(a) to 1.1(d) for performance differences of using the Fourier-based methods, Capon, and APES for spectral estimation). The fact that both Capon and APES make use of a set of finite impulse response (FIR) matched filters was observed in [70]. A number of results on the performance differences of the Capon and APES estimators were also reported therein. However, the study in [70] was somewhat limited since it only considered the forward-only (FO) Capon and APES estimators. Owing to the general belief that forward-backward (FB) approaches to spectral estimation usually provide more accurate results and are used more often than their FO counterparts, it is of interest to conduct a study that investigates how the FB Capon and APES spectral estimators perform when compared with one another as well as with their FO counterparts. In some sense, filterbank approaches reduce the problem of spectral estimation to a filter design problem subject to some constraints [46]. In Section 3.2, we discuss a general rule for choosing the impulse responses of the filterbank so that the signal-to- noise ratios (SNRs) at the outputs of the filterbank are maximized. It follows that the 91 Flop Ratio Â£ Exact Capon Approximate Capon M = K = 32 281 47 M = K = 50 655 82 II II a* 1020 115 Table 6.1: Comparison of the computational burdens of the standard and Capon methods with N = 256 and = 512. The approximate Capon method no longer needs to perform the spectral fac torization, and the computational demand is greatly reduced. The approximate Capon method requires approximately 2NM2 + 6M3 flops, a large part of which comes from the computation of the sample covariance matrix R. To illustrate quantitatively the computational burdens of the above three methods, we show a few simulation results in Table 6.1, where we define Â£ as the ratio of the flops needed by the exact or approximate Capon method to that corre sponding to the standard method. We remark that we did not pay special attention to the coding of the algorithms, hence the numbers provided here should be only taken as indicative of the computational complexities of the methods. 6.5 Numerical Results In this section, we present numerical examples showing the performance of the Capon method for covariance estimation. The first problem addressed is the ARMA covariance estimation. Several ARMA signals with different pole and zero locations are generated to compare the performances of the standard and the Capon methods. We also consider AR coefficient estimation of the ARMA signals by using the Capon and standard covariance estimates. Finally, we give an example illustrating the MA model determination with the standard and Capon methods. 11 that Capon spectral estimates do not possess the minimum variance property. In this study it is simply named the Capon spectral estimator. The Capon spectral estimator was originally introduced by Capon [16] [18] for applications in multidimensional seismic array frequency-wavenumber analysis. It is reformulated by Lacoss [44] for applications to 1-D time-series problems. The constraints adopted by the Capon method are such that the signal at the current frequency is passed undistorted (with unit gain) while the output power of the over all frequency domain is minimized. A number of researches have been conducted to study the Capon spectral estimator. Specifically, Capon and Goodman demon strated [19] [17] that the Capon spectrum has a mean and a variance that behaves like the averaged periodogram; that is, Capon spectra are usually less variable than periodograms. Additionally, Lacosss empirical study [44] suggested that the resolu tion of the Capon method is between that of Burgs AutoRegressive (AR) spectral estimator [9] [12] and that of the periodogram. Lacosss study also suggested that the statistical variability of Capon is less than that of the AR estimator. Burg later proved that the reciprocal of the Capon spectrum of order M, the length of the Capon filter, is equal to the average of the reciprocals of the AR spectra from order 0 to M [11]. Such an averaging effect explains the empirical observations made by Lacoss. Further researches in quantizing the resolution properties of Capon spectra were reported in [21] [58]. Since Capons method is a basic spectral estimator, it may be used in any ap plication where the spectrum of the studied signal plays an important role. Indeed, ever since its first appearance, the Capon spectral estimator has been widely used in many areas including radar, sonar, communications, imaging, geophysical explo ration, astrophysical exploration, and biomedical engineering (see [35] [79] and the references therein). Rather than making a futile attempt at making a full document 112 -a*(a"R-1aM)(a"R-ia)2 *(a^R-1aM)(a"R-1)(a"R"^) -a(a^R_1aM)2(//R-15) (a^R1aM)2(WR15)| = l-a{a.HM-R-1 *m)E {|a* R1^2 (a^R-1aM)(R~15)} (C.7) By the Cauchy-Schwartz inequality, the quantity between the curly parentheses in (C.7) is negative and so is its expectation. The bias for the FOC spectral estimate can be obtained by replacing the FB sample covariance Rpp in (C.1)-(C.3) by the FO sample covariance matrix Rpo> as defined in (3.12), and by following a similar treatment we did in (C.7). The result is as follows (to a second-order approximation): (a^R-1aM)2Â£ {FOC a} ~ {la^R-^f (a&R-VK^R"1*)} (C.8) Hence to within a second-order approximation, the bias of the FBC is one half that of the FOC, and (3.57) follows. Proof of (3.58): Again, we consider FBA first. Observe that QfbA Q 0(1/VZ). Then, by (3.34) and (B.13), we have (see (C.2)) FBA = smQfba^ aMQFBAaM a^Q-1(QFBA-Q)Q^+ a^Q-M amQ a^Q JaM (afcQ-^K CTHQfba Q)Q~laM (a^Q'^Ai)2 (C.9) Therefore, we obtain (awQ 1*m?E {AFba a} E {-(a^Q_1QFBAQ_15)(a^Q_1aM) + (a^Q1QFBAQ^1aM)(a^Q15)| . (C.10) |