Abstract
Toeplitz covariance matrices are used in the analysis of stationary stochastic processes and a wide range of applications including radar imaging, target detection, speech recognition, and communications systems. In this paper, we consider optimal estimation of large Toeplitz covariance matrices and establish the minimax rate of convergence for two commonly used parameter spaces under the spectral norm. The properties of the tapering and banding estimators are studied in detail and are used to obtain the minimax upper bound. The results also reveal a fundamental difference between the tapering and banding estimators over certain parameter spaces. The minimax lower bound is derived through a novel construction of a more informative experiment for which the minimax lower bound is obtained through an equivalent Gaussian scale model and through a careful selection of a finite collection of least favorable parameters. In addition, optimal rate of convergence for estimating the inverse of a Toeplitz covariance matrix is also established.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bentkus R.Yu., Rudzkis R.A.: On the distributions of some statistical estimates of spectral density. Theory Probab. Appl. 27, 795–814 (1982)
Bickel P.J., Levina E.: Regularized estimation of large covariance matrices. Ann. Stat. 36, 199–227 (2008)
Bickel P.J., Levina E.: Covariance regularization by thresholding. Ann. Stat. 36, 2577–2604 (2008)
Böttcher A., Silbermann B.: Introduction to Large Truncated Toeplitz Matrices. Springer, New York (1999)
Brockwell P.J., Davis R.A.: Time Series: Theory and Methods, 2nd edn. Springer, New York (1991)
Cai, T.T., Liu, W.: Adaptive thresholding for sparse covariance matrix estimation. J. Am. Stat. Assoc. (2011, to appear)
Cai, T.T., Liu, W., Luo, X.: A constrained ℓ 1 minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. (2010, to appear)
Cai, T.T., Liu, W., Zhou, H.H.: Optimal estimation of large sparse precision matrices (2010, manuscript)
Cai T.T., Zhang C.-H., Zhou H.H.: Optimal rates of convergence for covariance matrix estimation. Ann. Stat. 38, 2118–2144 (2010)
Cai, T.T., Zhou, H.H.: Optimal rates of convergence for sparse covariance matrix estimation. Technical report (2010)
Cai, T.T., Zhou, H.H.: Minimax estimation of large covariance matrices under ℓ 1 norm. Statistica Sinica (2011, to appear)
Christensen, L.P.B.: An EM-algorithm for band-toeplitz covariance matrix estimation. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Honolulu (2007)
El Karoui N.: Operator norm consistent estimation of large dimensional sparse covariance matrices. Ann. Stat. 36, 2717–2756 (2008)
Fuhrmann D.R.: Application of Toeplitz covariance estimation to adaptive beamforming and detection. IEEE Trans. Signal Process. 39, 2194–2198 (1991)
Golubev G.K., Nussbaum M., Zhou H.H.: Asymptotic equivalence of spectral density estimation and Gaussian white noise. Ann. Stat. 38, 181–214 (2010)
Lam C., Fan J.: Sparsistency and rates of convergence in large covariance matrices estimation. Ann. Stat. 37, 4254–4278 (2009)
Parzen E.: On consistent estimates of the spectrum of a stationary time series. Ann. Math. Stat. 28, 329–348 (1957)
Petrov V.V.: Sums of Independent Random Variables. Springer, New York (1975)
Politis D.N., Romano J.P.: Bias-corrected nonparametric spectral estimation. J. Time Ser. Anal. 16, 67–103 (1995)
Priestley M.B.: Basic considerations in the estimation of spectra. Technometrics 4, 551–564 (1962)
Ravikumar, P., Wainwright, M.J., Raskutti, G., Yu, B.: High-dimensional covariance estimation by minimizing l 1 -penalized log-determinant divergence. Technical Report, UC Berkeley (2008)
Roberts W.J., Ephraim Y.: Hidden Markov modeling of speech using Toeplitz covariance matrices. Speech Commun. 31, 1–14 (2000)
Samarov A.: Lower bound for the risk of spectral density estimates. Probl. Inform. Transm. 13, 67–72 (1977)
Snyder D.L, O’Sullivan J.A., Miller M.I.: The use of maximum-likelihood estimation for forming images of diffuse radar-targets from delay-Doppler data. IEEE Trans. Inform. Theory 35, 536–548 (1989)
Tsybakov A.B.: Introduction to Nonparametric Estimation. Springer, New York (2009)
Woodroofe M.B., Van Ness J.W.: The maximum deviation of sample spectral densities. Ann. Math. Stat. 38, 1558–1569 (1967)
Yuan M.: Sparse inverse covariance matrix estimation via linear programming. J. Mach. Learn. Res. 11, 2261–2286 (2010)
Zygmund A.: Trigonometric series. vols. I, II. Cambridge Mathematical Library, 3rd edn., Cambridge University Press, Cambridge (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
The research of T. Cai was supported in part by NSF FRG Grant DMS-0854973 and the research of Z. Ren and H. Zhou was supported in part by NSF Career Award DMS-0645676 and NSF FRG Grant DMS-0854975.
Rights and permissions
About this article
Cite this article
Cai, T.T., Ren, Z. & Zhou, H.H. Optimal rates of convergence for estimating Toeplitz covariance matrices. Probab. Theory Relat. Fields 156, 101–143 (2013). https://doi.org/10.1007/s00440-012-0422-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00440-012-0422-7
Keywords
- Banding
- Covariance matrix
- Minimax lower bound
- Optimal rate of convergence
- Spectral norm
- Tapering
- Toeplitz covariance matrix