Second Order Reverse Mode
Keywords: second order reverse mode derivative syntax purpose x^(k) w f dw first partials vector hessian times direction example

| Prev | Next |  |  |  |  |  |  |

Second Order Reverse Mode
Syntax
`dw = f.Reverse(2, w)`
Purpose
We use F : B n → B m to denote the AD function corresponding to `f` . Reverse mode computes the derivative of the Forward mode Taylor coefficients with respect to the domain variable x .
x^(k)
For k = 0 , 1 , the vector x ( k ) ∈ B n is defined as the value of `x_k` in the previous call (counting this call) of the form `
f.Forward(k, x_k)
` If there is no previous call with k = 0 , x ( 0 ) is the value of the independent variables when the corresponding AD of `Base` operation sequence was recorded.
W
The functions W 0 : B n → B and W 1 : B n → B are defined by W 0 ( u ) = w 0 * F 0 ( u ) + ⋯ + w m -1 * F m -1 ( u ) W 1 ( u ) = w 0 * F 0 ( 1 ) ( u ) * x ( 1 ) + ⋯ + w m -1 * F m -1 ( 1 ) ( u ) * x ( 1 ) This operation computes the derivatives W 0 ( 1 ) ( u ) = w 0 * F 0 ( 1 ) ( u ) + ⋯ + w m -1 * F m -1 ( 1 ) ( u ) W 1 ( 1 ) ( u ) = w 0 * (x ( 1 ) ) T * F 0 ( 2 ) ( u ) + ⋯ + w m -1 * (x ( 1 ) ) T F m -1 ( 2 ) ( u ) at u = x ( 0 ) .
f
The object `f` has prototype `
const ADFun<Base> f
` Before this call to `Reverse`, the value returned by `
f.size_order()
` must be greater than or equal two (see size_order ).
w
The argument `w` has prototype `
const Vector &w
` (see Vector below) and its size must be equal to `m` , the dimension of the range space for `f` .
dw
The result `dw` has prototype `
Vector dw
` (see Vector below). It contains both the derivative W ( 1 ) ( x ) and the derivative U ( 1 ) ( x ) . The size of `dw` is equal to n × 2 , where n is the dimension of the domain space for `f` .
First Order Partials
For j = 0 , … , n - 1 , dw [ j * 2 + 0 ] = ∂ W 0 ∂ u j (x ( 0 ) ) = w 0 * ∂ F 0 ∂ u j (x ( 0 ) ) + ⋯ + w m -1 * ∂ F m -1 ∂ u j (x ( 0 ) ) This part of `dw` contains the same values as are returned by reverse_one .
Second Order Partials
For j = 0 , … , n - 1 , dw [ j * 2 + 1 ] = ∂ W 1 ∂ u j (x ( 0 ) ) = ∑ ℓ = 0 n -1 x ℓ ( 1 ) [w 0 * ∂ 2 F 0 ∂ u ℓ ∂ u j (x ( 0 ) ) + ⋯ + w m -1 * ∂ 2 F m -1 ∂ u ℓ ∂ u j (x ( 0 ) ) ]
Vector
The type `Vector` must be a SimpleVector class with elements of type `Base` . The routine CheckSimpleVector will generate an error message if this is not the case.
Hessian Times Direction
Suppose that w is the `i`-th elementary vector. It follows that for j = 0 , … , n -1 dw [ j * 2 + 1 ] = w i ∑ ℓ = 0 n -1 ∂ 2 F i ∂ u j ∂ u ℓ (x ( 0 ) ) x ℓ ( 1 ) = [F i ( 2 ) (x ( 0 ) ) * x ( 1 ) ] j Thus the vector ( dw [ 1 ] , dw [ 3 ] , … , dw [ n * q - 1 ] ) is equal to the Hessian of F i ( x ) times the direction x ( 1 ) . In the special case where x ( 1 ) is the `l`-th elementary vector , dw [ j * 2 + 1 ] = ∂ 2 F i ∂ x j ∂ x ℓ (x ( 0 ) )
Example
The files reverse_two.cpp and hes_times_dir.cpp contain a examples and tests of reverse mode calculations. They return true if they succeed and false otherwise.

Input File: omh/reverse/reverse_two.omh