Abstract
Partial Least Squares (PLS) is a wide class of methods for modeling relations between sets of observed variables by means of latent variables. It comprises of regression and classification tasks as well as dimension reduction techniques and modeling tools. The underlying assumption of all PLS methods is that the observed data is generated by a system or process which is driven by a small number of latent (not directly observed or measured) variables. Projections of the observed data to its latent structure by means of PLS was developed by Herman Wold and coworkers [48,49,52].
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Mean Square Error
- Partial Little Square
- Partial Little Square Regression
- Canonical Correlation Analysis
- Principal Component Regression
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Almøy, T.: A simulation study on comparison of prediction models when only a few components are relevant. Computational Statistics and Data Analysis 21, 87–107 (1996)
Aronszajn, N.: Theory of reproducing kernels. Transactions of the American Mathematical Society 68, 337–404 (1950)
Baffi, G., Martin, E.B., Morris, A.J.: Non-linear projection to latent structures revisited (the neural network PLS algorithm). Computers Chemical Engineering 23, 1293–1307 (1999)
Barker, M., Rayens, W.S.: Partial least squares for discrimination. Journal of Chemometrics 17, 166–173 (2003)
Bartlett, M.S.: Further aspects of the theory of multiple regression. In: Proceedings of the Cambridge Philosophical Society, vol. 34, pp. 33–40 (1938)
De Bie, T., Cristianini, N., Rosipal, R.: Eigenproblems in Pattern Recognition. In: Bayro-Corrochano, E. (ed.) Handbook of Geometric Computing: Applications in Pattern Recognition, Computer Vision, Neuralcomputing, and Robotics, pp. 129–170. Springer, Heidelberg (2005)
Butler, N.A., Denham, M.C.: The peculiar shrinkage properties of partial least squares regression. Journal of the Royal Statistical Society: B 62, 585–593 (2000)
de Jong, S.: SIMPLS: an alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems 18, 251–263 (1993)
de Jong, S., Wise, B.M., Ricker, N.L.: Canonical partial least squares and continuum power regression. Journal of Chemometrics 15, 85–100 (2001)
Frank, I.E.: A nonlinear PLS model. Chemolab 8, 109–119 (1990)
Frank, I.E., Friedman, J.H.: A Statistical View of Some Chemometrics Regression Tools. Technometrics 35, 109–147 (1993)
Goutis, C.: Partial least squares yields shrinkage estimators. The Annals of Statistics 24, 816–824 (1996)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)
Helland, I.S.: Some theoretical aspects of partial least squares regression. Chemometrics and Intelligent Laboratory Systems 58, 97–107 (1999)
Hestenes, M., Stiefel, E.: Methods for conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards 49, 409–436 (1952)
Hoerl, A.E., Kennard, R.W.: Ridge regression: bias estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)
Höskuldsson, A.: PLS Regression Methods. Journal of Chemometrics 2, 211–228 (1988)
Hulland, J.S.: Use of partial least squares (PLS) in strategic management research: A review of four recent studies. Strategic Management Journal 20, 195–204 (1999)
Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards 45, 225–280 (1950)
Lewi, P.J.: Pattern recognition, reflection from a chemometric point of view. Chemometrics and Intelligent Laboratory Systems 28, 23–33 (1995)
Lingjærde, O.C., Christophersen, N.: Shrinkage Structure of Partial Least Squares. Scandinavian Journal of Statistics 27, 459–473 (2000)
Lobaugh, N.J., West, R., McIntosh, A.R.: Spatiotemporal analysis of experimental differences in event-related potential data with partial least squares. Psychophysiology 38, 517–530 (2001)
Manne, R.: Analysis of Two Partial-Least-Squares Algorithms for Multivariate Calibration. Chemometrics and Intelligent Laboratory Systems 2, 187–197 (1987)
Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic Press, London (1997)
Martens, M., Martens, H.: Partial Least Squares Regression. In: Piggott, J.R. (ed.) Statistical Procedures in Food Research, pp. 293–359. Elsevier Applied Science, London (1986)
Massy, W.F.: Principal components regression in exploratory statistical research. Journal of the American Statistical Association 60, 234–256 (1965)
Momma, M.: Efficient Computations via Scalable Sparse Kernel Partial Least Squares and Boosted Latent Features. In: Proceedings of SIGKDD International Conference on Knowledge and Data Mining, Chicago, IL, pp. 654–659 (2005)
Nguyen, D.V., Rocke, D.M.: Tumor classification by partial least squares using microarray gene expression data. Bioinformatics 18, 39–50 (2002)
Nilsson, J., de Jong, S., Smilde, A.K.: Multiway Calibration in 3D QSAR. Journal of Chemometrics 11, 511–524 (1997)
Parlett, B.: The symmetric eigenvalue problem. SIAM, Philadelphia (1998)
Phatak, A., de Hoog, F.: Exploiting the connection between PLS, Lanczos, and conjugate gradients: Alternative proofs of some properties of PLS. Journal of Chemometrics 16, 361–367 (2003)
Phatak, A., Rilley, P.M., Penlidis, A.: The asymptotic variance of the univariate PLS estimator. Linear Algebra and its Applications 354, 245–253 (2002)
Rännar, S., Lindgren, F., Geladi, P., Wold, S.: A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and algorithm. Chemometrics and Intelligent Laboratory Systems 8, 111–125 (1994)
Rosipal, R., Trejo, L.J.: Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. Journal of Machine Learning Research 2, 97–123 (2001)
Rosipal, R., Trejo, L.J., Matthews, B.: Kernel PLS-SVC for Linear and Nonlinear Classification. In: Proceedings of the Twentieth International Conference on Machine Learning, Washington, DC, pp. 640–647 (2003)
Sampson, P.D., Streissguth, A.P., Barr, H.M., Bookstein, F.L.: Neurobehavioral effects of prenatal alcohol: Part II. Partial Least Squares analysis. Neurotoxicology and tetralogy 11, 477–491 (1989)
Saunders, C., Gammerman, A., Vovk, V.: Ridge Regression Learning Algorithm in Dual Variables. In: Proceedings of the 15th International Conference on Machine Learning, Madison, WI, pp. 515–521 (1998)
Schölkopf, B., Smola, A.J.: Learning with Kernels – Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press, Cambridge (2002)
Smilde, A., Bro, R., Geladi, P.: Multi-way Analysis: Applications in the Chemical Sciences. Wiley, Chichester (2004)
Stone, M., Brooks, R.J.: Continuum Regression: Cross-validated Sequentially Constructed Prediction Embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression. Journal of the Royal Statistical Society: B 52, 237–269 (1990)
Vinod, H.D.: Canonical ridge and econometrics of joint production. Journal of Econometrics 4, 147–166 (1976)
Wahba, G.: Splines Models of Observational Data. Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990)
Wallerius, J., Trejo, L.J., Matthew, R., Rosipal, R., Caldwell, J.A.: Robust feature extraction and classification of EEG spectra for real-time classification of cognitive state. In: Proceedings of 11th International Conference on Human Computer Interaction, Las Vegas, NV (2005)
Wangen, L.E., Kowalsky, B.R.: A multiblock partial least squares algorithm for investigating complex chemical systems. Journal of Chemometrics 3, 3–20 (1989)
Wegelin, J.A.: A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case. Technical report, Department of Statistics, University of Washington, Seattle (2000)
Westerhuis, J., Kourti, T., MacGregor, J.: Analysis of multiblock and hierarchical PCA and PLS models. Journal of Chemometrics 12, 301–321 (1998)
Wold, H.: Path models with latent variables: The NIPALS approach. In: Blalock, H.M., et al. (eds.) Quantitative Sociology: International perspectives on mathematical and statistical model building, pp. 307–357. Academic Press, London (1975)
Wold, H.: Soft modeling: the basic design and some extensions. In: Jöreskog, J.-K., Wold, H. (eds.) Systems Under Indirect Observation, vol. 2, pp. 1–53. North Holland, Amsterdam (1982)
Wold, H.: Partial least squares. In: Kotz, S., Johnson, N.L. (eds.) Encyclopedia of the Statistical Sciences, vol. 6, pp. 581–591. John Wiley & Sons, Chichester (1985)
Wold, S.: Nonlinear partial least squares modeling II, Spline inner relation. Chemolab 14, 71–84 (1992)
Wold, S., Kettaneh-Wold, N., Skagerberg, B.: Nonlinear PLS modelling. Chemometrics and Intelligent Laboratory Systems 7, 53–65 (1989)
Wold, S., Ruhe, H., Wold, H., Dunn III., W.J.: The collinearity problem in linear regression. The partial least squares (PLS) approach to generalized inverse. SIAM Journal of Scientific and Statistical Computations 5, 735–743 (1984)
Worsley, K.J.: An overview and some new developments in the statistical analysis of PET and fMRI data. Human Brain Mapping 5, 254–258 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rosipal, R., Krämer, N. (2006). Overview and Recent Advances in Partial Least Squares. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds) Subspace, Latent Structure and Feature Selection. SLSFS 2005. Lecture Notes in Computer Science, vol 3940. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11752790_2
Download citation
DOI: https://doi.org/10.1007/11752790_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34137-6
Online ISBN: 978-3-540-34138-3
eBook Packages: Computer ScienceComputer Science (R0)