Optimization in regression problems

Abstract

Many optimization techniques described in the previous chapters demand large amounts of arithmetic, and their routine application to many statistical problems has only become possible with the advent of more powerful computers. For example, it is now over 40 years since Lawley formulated the maximum likelihood approach to factor analysis (Lawley, 1940), but its use has only become routine since the late 1960s when Karl Jöreskog took advantage of the development of more efficient optimization algorithms allied with the availability of the electronic computer, to produce a practical method of estimation. This, and other examples where the combination of computer power and improvements in optimization methods have made particular statistical techniques practical possibilities, will be the subject of the next two chapters. In this chapter we shall concentrate on developments in regression methods, and in Chapter 6 methods of multivariate analysis will be considered.