Author	Hestenes, Magnus Rudolph. author
Title	Conjugate Direction Methods in Optimization [electronic resource] / by Magnus Rudolph Hestenes
Imprint	New York, NY : Springer New York, 1980
Connect to	http://dx.doi.org/10.1007/978-1-4612-6048-6
Descript	X, 325 p. online resource

Shortly after the end of World War II high-speed digital computing machines were being developed. It was clear that the mathematical aspects of comยญ putation needed to be reexamined in order to make efficient use of high-speed digital computers for mathematical computations. Accordingly, under the leadership of Min a Rees, John Curtiss, and others, an Institute for Numerical Analysis was set up at the University of California at Los Angeles under the sponsorship of the National Bureau of Standards. A similar institute was formed at the National Bureau of Standards in Washington, D. C. In 1949 J. Barkeley Rosser became Director of the group at UCLA for a period of two years. During this period we organized a seminar on the study of soluยญ tions of simultaneous linear equations and on the determination of eigenยญ values. G. Forsythe, W. Karush, C. Lanczos, T. Motzkin, L. J. Paige, and others attended this seminar. We discovered, for example, that even Gausยญ sian elimination was not well understood from a machine point of view and that no effective machine oriented elimination algorithm had been developed. During this period Lanczos developed his three-term relationship and I had the good fortune of suggesting the method of conjugate gradients. We disยญ covered afterward that the basic ideas underlying the two procedures are essentially the same. The concept of conjugacy was not new to me. In a joint paper with G. D

I Newtonโ{128}{153}s Method and the Gradient Method -- 1 Introduction -- 2 Fundamental Concepts -- 3 Iterative Methods for Solving g(x) = 0 -- 4 Convergence Theorems -- 5 Minimization of Functions by Newtonโ{128}{153}s Method -- 6 Gradient Methodsโ{128}{148}The Quadratic Case -- 7 General Descent Methods -- 8 Iterative Methods for Solving Linear Equations -- 9 Constrained Minima -- II Conjugate Direction Methods -- 1 Introduction -- 2 Quadratic Functions on En -- 3 Basic Properties of Quadratic Functions -- 4 Minimization of a Quadratic Function F on k-Planes -- 5 Method of Conjugate Directions (CD-Method) -- 6 Method of Conjugate Gradients (CG-Algorithm) -- 7 Gradient PARTAN -- 8 CG-Algorithms for Nonquadratic Functions -- 9 Numerical Examples -- 10 Least Square Solutions -- III Conjugate Gram-Schmidt Processes -- 1 Introduction -- 2 A Conjugate Gram-Schmidt Process -- 3 CGS-CG-Algorithms -- 4 A Connection of CGS-Algorithms with Gaussian Elimination -- 5 Method of Parallel Displacements -- 6 Methods of Parallel Planes (PARP) -- 7 Modifications of Parallel Displacements Algorithms -- 8 CGS-Algorithms for Nonquadratic Functions -- 9 CGS-CG-Routines for Nonquadratic Functions -- 10 Gauss-Seidel CGS-Routines -- 11 The Case of Nonnegative Components -- 12 General Linear Inequality Constraints -- IV Conjugate Gradient Algorithms -- 1 Introduction -- 2 Conjugate Gradient Algorithms -- 3 The Normalized CG-Algorithm -- 4 Termination -- 5 Clustered Eigenvalues -- 6 Nonnegative Hessians -- 7 A Planar CG-Algorithm -- 8 Justification of the Planar CG-Algorithm -- 9 Modifications of the CG-Algorithm -- 10 Two Examples -- 11 Connections between Generalized CG-Algorithms and Stadard CG- and CD-Algorithm -- 12 Least Square Solutions -- 13 Variable Metric Algorithms -- 14 A Planar CG-Algorithm for Nonquadratic Functions -- References