Unifying Least Squares, Total Least Squares and Data Least Squares

Abstract

The standard approaches to solving overdetermined linear systems A _x ≈ b construct minimal corrections to the vector b and/or the matrix A such that the corrected system is compatible. In ordinary least squares (LS) the correction is restricted to b, while in data least squares (DLS) it is restricted to A. In scaled total least squares (Scaled TLS) [15], corrections to both b and A are allowed, and their relative sizes depend on a parameter γ. Scaled TLS becomes total least squares (TLS) when γ = 1, and in the limit corresponds to LS when γ → 0, and DLS when γ → ∞.

In [13] we presented a particularly useful formulation of the Scaled TLS problem, as well as a new assumption that guarantees the existence and uniqueness of meaningful Scaled TLS solutions for all parameters γ > 0, making the whole Scaled TLS theory consistent. This paper refers to results in [13] and is mainly historical, but it also gives some simpler derivations and some new theory. Here it is shown how any linear system Ax ≈ b can be reduced to a minimally dimensioned core system satisfying our assumption. The basics of practical algorithms for both the Scaled TLS and DLS problems are indicated for either dense or large sparse systems.