GNU Scientific Library – Reference Manual: Nonlinear Least-Squares TRS Levenberg-Marquardt with Geodesic Acceleration

Next: Nonlinear Least-Squares TRS Dogleg, Previous: Nonlinear Least-Squares TRS Levenberg-Marquardt, Up: Nonlinear Least-Squares TRS Overview [Index]

#### 39.2.2 Levenberg-Marquardt with Geodesic Acceleration

This method applies a so-called geodesic acceleration correction to the standard Levenberg-Marquardt step \delta_k (Transtrum et al, 2011). By interpreting \delta_k as a first order step along a geodesic in the model parameter space (ie: a velocity \delta_k = v_k), the geodesic acceleration a_k is a second order correction along the geodesic which is determined by solving the linear least squares system

    [J_k; sqrt(mu_k) D_k] a_k = - [f_vv(x_k); 0]

where f_{vv} is the second directional derivative of the residual vector in the velocity direction v, f_{vv}(x) = D_v^2 f = \sum_{\alpha\beta} v_{\alpha} v_{\beta} \partial_{\alpha} \partial_{\beta} f(x), where \alpha and \beta are summed over the p parameters. The new total step is then \delta_k' = v_k + {1 \over 2}a_k. The second order correction a_k can be calculated with a modest additional cost, and has been shown to dramatically reduce the number of iterations (and expensive Jacobian evaluations) required to reach convergence on a variety of different problems. In order to utilize the geodesic acceleration, the user must supply a function which provides the second directional derivative vector f_{vv}(x), or alternatively the library can use a finite difference method to estimate this vector with one additional function evaluation of f(x + h v) where h is a tunable step size (see the `h_fvv` parameter description).