Abstract
We establish a Karhunen-Loève expansion for generic centered, second order stochastic processes, which does not rely on topological assumptions. We further investigate in which norms the expansion converges and derive exact average rates of convergence for these norms. For Gaussian processes as well as for some other processes we additionally prove certain sharpness results in terms of the norm. Moreover, we investigate when the generic Karhunen-Loève expansion can be used to construct reproducing kernel Hilbert spaces (RKHSs) containing the paths of a version of the process. We further illustrate how the general theory can be applied, even in the absence of an explicitly known Karhunen-Loève expansion, by comparing the smoothness of the paths with the smoothness of the functions contained in the RKHS of the covariance function and by discussing some small ball probabilities. Key tools for our results are a recently shown generalization of Mercer’s theorem, spectral properties of the covariance integral operator, interpolation spaces of the real method, and compactness results for embeddings between classical function spaces.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Adams, R.A., Fournier, J.J.F.: Sobolev Spaces, 2nd edn. Academic Press, New York (2003)
Adler, R.J.: An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes. Hayward, Institute of Mathematical Statistics (1990)
Ai, X., Li, W.V., Liu, G.: Karhunen-Loève expansions for the detrended Brownian motion. Statist. Probab. Lett. 82, 1235–1241 (2012)
Aurzada, F.: On the lower tail probabilities of some random sequences in ℓ p. J. Theor. Probab. 20, 843–858 (2007)
Barczy, M., Iglói, E.: Karhunen-Loève expansions of α-Wiener bridges. Cent. Eur. J. Math. 9, 65–84 (2011)
Bennett, C., Sharpley, R.: Interpolation of Operators. Academic Press, Boston (1988)
Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer, Boston (2004)
Birman, M.S.h., Solomjak, M.Z.: Spectral Theory of Selfadjoint Operators in Hilbert Space D. Reidel Publishing Co., Dordrecht (1987)
Borovkov, A.A., Ruzankin, P.S.: On small deviations of series of weighted random variables. J. Theor. Probab. 21, 628–649 (2008)
Bozzini, M., Rossini, M., Schaback, R.: Generalized Whittle-Matérn and polyharmonic kernels. Adv. Comput. Math. 39, 129–141 (2013)
Bronski, J.C.: Small ball constants and tight eigenvalue asymptotics for fractional Brownian motions. J. Theor. Probab. 16, 87–100 (2003)
Cramér, H., Leadbetter, M.R.: Stationary and Related Stochastic Processes. Sample Function Properties and their Applications. Wiley, New York (1967)
Deheuvels, P.: Karhunen-Loève expansions of mean-centered Wiener processes. In: High Dimensional Probability, Volume 51 of IMS Lecture Notes Monogr. Ser., pp. 62–76. Inst. Math. Statist., Beachwood (2006)
Deheuvels, P.: A Karhunen-Loève expansion for a mean-centered Brownian bridge. Statist. Probab. Lett. 77, 1190–1200 (2007)
Deheuvels, P., Martynov, G.V.: A Karhunen-Loève decomposition of a Gaussian process generated by independent pairs of exponential random variables. J. Funct. Anal. 255, 2363–2394 (2008)
Deheuvels, P., Peccati, G., Yor, M.: On quadratic functionals of the Brownian sheet and related processes. Stoch. Process. Appl. 116, 493–538 (2006)
Devore, R.A., Sharpley, R.C.: Besov spaces on domains in \(\mathbb {R}^{d}\). Trans. Am. Math. Soc. 335, 843–864 (1993)
Diestel, J., Uhl, J.J.: Vector Measures. American Mathematical Society, Providence (1977)
Dinculeanu, N.: Vector Integration and Stochastic Integration in Banach Spaces. Wiley, New York (2000)
Edmunds, D.E., Triebel, H.: Function Spaces, Entropy Numbers, Differential Operators. Cambridge University Press, Cambridge (1996)
Gao, F., Hannig, J., Lee, T.-Y., Torcaso, F.: Exact L 2 small balls of Gaussian processes. J. Theor. Probab. 17, 503–520 (2004)
Gao, F., Hannig, J., Torcaso, F.: Integrated Brownian motions and exact L 2-small balls. Ann. Probab. 31, 1320–1337 (2003)
Gneiting, T., Kleiber, W., Schlather, M.: Matérn cross-covariance functions for multivariate random fields. J. Amer. Statist. Assoc. 105, 1167–1177 (2010)
Handcock, M.S., Stein, M.L.: A Bayesian analysis of Kriging. Technometrics 35, 403–410 (1993)
Herren, V.: Lévy-type processes and Besov spaces. Potential Anal. 7, 689–704 (1997)
Hoang, V.H., Schwab, C.: Analytic regularity and polynomial approximation of stochastic, parametric elliptic multiscale PDEs. Anal. Appl. (Singap.), 11 (2013)
Istas, J.: Karhunen-Loève expansion of spherical fractional Brownian motions. Statist. Probab. Lett. 76, 1578–1583 (2006)
Janson, S.: Gaussian Hilbert Spaces. Cambridge University Press, Cambridge (1997)
Karol’, A., Nazarov, A., Nikitin, Y.: Small ball probabilities for Gaussian random fields and tensor products of compact operators. Trans. Am. Math. Soc. 360, 1443–1474 (2008)
Karol’, A.I., Nazarov, A.I.: Small ball probabilities for smooth Gaussian fields and tensor products of compact operators. Math. Nachr. 287, 595–609 (2014)
Kelley, J.L.: General Topology. D. Van Nostrand, Toronto (1955)
Klenke, A.: Probability Theory. Springer, London (2008)
Lang, A., Schwab, C.: Isotropic Gaussian random fields on the sphere: regularity, fast simulation, and stochastic partial differential equations Technical Report 2013–15. ETH Zürich, Seminar for Applied Mathematics (2013)
Li, W.V., Shao, Q.-M.: Small ball estimates for Gaussian processes under Sobolev type norms. J. Theor. Probab. 12, 699–720 (1999)
Li, W.V., Shao, Q.-M.: Gaussian processes: Inequalities, small ball probabilities and applications. In: Stochastic Processes: Theory and Methods, Volume 19 of Handbook of Statist., pp. 533–597. North-Holland, Amsterdam (2001)
Lifshits, M.: Lectures on Gaussian Processes. Springer, Heidelberg (2012)
Lifshits, M., Papageorgiou, A., Woźniakowski, H.: Tractability of multi-parametric Euler and Wiener integrated processes. Probab. Math. Statist. 32, 131–165 (2012)
Lifshits, M.A., Papageorgiou, A., Woźniakowski, H.: Average case tractability of non-homogeneous tensor product problems. J. Complexity 28, 539–561 (2012)
Liu, J.V.: Karhunen-Loève expansion for additive Brownian motions. Stoch. Process. Appl. 123, 4090–4110 (2013)
Liu, J.V., Huang, Z., Mao, H.: Karhunen–Loève expansion for additive Slepian processes. Statist. Probab. Lett. 90, 93–99 (2014)
Lord, G.J., Powell, C.E., Shardlow, T.: An Introduction to Computational Stochastic PDEs. Cambridge University Press, Cambridge (2014)
Lukić, M.N.: Integrated Gaussian processes and their reproducing kernel Hilbert spaces. In: Stochastic Processes and Functional Analysis, pp. 241–263. Dekker, New York (2004)
Lukić, M.N., Beder, J.H.: Stochastic processes with sample paths in reproducing kernel Hilbert spaces. Trans. Amer. Math Soc. 353, 3945–3969 (2001)
Megginson, R.E.: An Introduction to Banach Space Theory. Springer, New York (1998)
Nazarov, A.I., Nikitin, Y.Y.: Exact L 2-small ball behavior of integrated Gaussian processes and spectral asymptotics of boundary value problems. Probab. Theory Relat. Fields 129, 469–494 (2004)
Parzen, E.: An appproach to time series analysis. Ann. Math. Statist. 32, 951–989 (1961)
Pillai, N.S., Wu, Q., Liang, F., Mukherjee, S., Wolpert, R.L.: Characterizing the function space for Bayesian kernel models. J. Mach. Learn Res. 8, 1769–1797 (2007)
Pycke, J.-R.: Une généralisation du développement de Karhunen-Loève du pont brownien. C. R. Acad. Sci. Paris Sér. I Math. 333, 685–688 (2001)
Pycke, J.-R.: U-statistics based on the Green’s function of the Laplacian on the circle and the sphere. Statist. Probab. Lett. 77, 863–872 (2007)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Schaback, R., Wendland, H.: Characterization and construction of radial basis functions. In: Multivariate Approximation and Applications, pp. 1–24. Cambridge University Press, Cambridge (2001)
Scheuerer, M.: Regularity of the sample paths of a general second order random field. Stoch. Process. Appl. 120, 1879–1897 (2010)
Scheuerer, M., Schaback, R., Schlather, M.: Interpolation of spatial data—a stochastic or a deterministic problem? European J. Appl. Math. 24, 601–629 (2013)
Schwab, C., Todor, R.A.: Karhunen-Loève approximation of random fields by generalized fast multipole methods. J. Comput. Phys. 217, 100–122 (2006)
Stein, M.L.: Interpolation of Spatial Data. Springer, New York (1999)
Steinwart, I.: Convergence types and rates in generic Karhunen-Loève expansions with applications to sample path properties. Technical Report arXiv:1403.1040 (2014)
Steinwart, I.: A short note on the comparison of interpolation widths, entropy numbers, and Kolmogorov widths. J. Approx. Theory 215, 13–17 (2017)
Steinwart, I., Christmann, A.: Support Vector Machines. Springer, New York (2008)
Steinwart, I., Scovel, C.: Mercer’s theorem on general domains: on the interaction between measures, kernels, and RKHSs. Constr. Approx. 35, 363–417 (2012)
Stolz, W.: Une méthode élémentaire pour l’évaluation de petites boules browniennes. C. R. Acad. Sci. Paris Sér. I Math. 316, 1217–1220 (1993)
Tartar, L.: An Introduction to Sobolev Spaces and Interpolation Spaces. Springer, Berlin (2007)
Ullrich, D.C.: Besov spaces: A primer. Technical report. https://www.math.okstate.edu/ullrich/besov/besov.pdf
van der Vaart, A., van Zanten, H.: Information rates of nonparametric Gaussian process methods. J. Mach. Learn. Res. 12, 2095–2119 (2011)
van der Vaart, A.W., van Zanten, J.H.: Reproducing kernel Hilbert spaces of Gaussian priors. In: Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, pp. 200–222. Inst. Math. Statist., Beachwood (2008)
Wendland, H.: Scattered Data Approximation. Cambridge University Press, Cambridge (2005)
Wojtaszczyk, P.: Banach Spaces for Analysts. Cambridge University Press, Cambridge (1991)
Xu, G.: Quasi-polynomial tractability of linear problems in the average case setting. J. Complex. 30, 54–68 (2014)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Steinwart, I. Convergence Types and Rates in Generic Karhunen-Loève Expansions with Applications to Sample Path Properties. Potential Anal 51, 361–395 (2019). https://doi.org/10.1007/s11118-018-9715-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11118-018-9715-5