// $Id: reverse_two.omh 3757 2015-11-30 12:03:07Z bradbell $
/* --------------------------------------------------------------------------
CppAD: C++ Algorithmic Differentiation: Copyright (C) 2003-15 Bradley M. Bell

CppAD is distributed under multiple licenses. This distribution is under
the terms of the
                    GNU General Public License Version 3.

A copy of this license is included in the COPYING file of this distribution.
Please visit http://www.coin-or.org/CppAD/ for information on other licenses.
-------------------------------------------------------------------------- */
$begin reverse_two$$

$section Second Order Reverse Mode$$
$mindex derivative$$
$spell
	taylor
	const
	dw
$$


$head Syntax$$
$icode%dw% = %f%.Reverse(2, %w%)%$$

$head Purpose$$
We use $latex F : B^n \rightarrow B^m$$ to denote the
$cref/AD function/glossary/AD Function/$$ corresponding to $icode f$$.
Reverse mode computes the derivative of the $cref Forward$$ mode
$cref/Taylor coefficients/glossary/Taylor Coefficient/$$
with respect to the domain variable $latex x$$.

$head x^(k)$$
For $latex k = 0, 1$$,
the vector $latex x^{(k)} \in B^n$$ is defined as the value of
$icode x_k$$ in the previous call (counting this call) of the form
$codei%
	%f%.Forward(%k%, %x_k%)
%$$
If there is no previous call with $latex k = 0$$,
$latex x^{(0)}$$ is the value of the independent variables when the
corresponding
AD of $icode Base$$
$cref/operation sequence/glossary/Operation/Sequence/$$ was recorded.


$head W$$
The functions
$latex W_0 : B^n \rightarrow B$$ and
$latex W_1 : B^n \rightarrow B$$
are defined by
$latex \[
\begin{array}{rcl}
W_0 ( u ) & = & w_0 * F_0 ( u ) + \cdots + w_{m-1} * F_{m-1} (u)
\\
W_1 ( u ) & = &
w_0 * F_0^{(1)} ( u ) * x^{(1)}
	+ \cdots + w_{m-1} * F_{m-1}^{(1)} (u) * x^{(1)}
\end{array}
\] $$
This operation computes the derivatives
$latex \[
\begin{array}{rcl}
W_0^{(1)} (u) & = &
	w_0 * F_0^{(1)} ( u ) + \cdots + w_{m-1} * F_{m-1}^{(1)} (u)
\\
W_1^{(1)} (u) & = &
	w_0 * \left( x^{(1)} \right)^\R{T} * F_0^{(2)} ( u )
	+ \cdots +
	w_{m-1} * \left( x^{(1)} \right)^\R{T} F_{m-1}^{(2)} (u)
\end{array}
\] $$
at $latex u = x^{(0)}$$.

$head f$$
The object $icode f$$ has prototype
$codei%
	const ADFun<%Base%> %f%
%$$
Before this call to $code Reverse$$, the value returned by
$codei%
	%f%.size_order()
%$$
must be greater than or equal two (see $cref size_order$$).

$head w$$
The argument $icode w$$ has prototype
$codei%
	const %Vector% &%w%
%$$
(see $cref/Vector/reverse_two/Vector/$$ below)
and its size
must be equal to $icode m$$, the dimension of the
$cref/range/seq_property/Range/$$ space for $icode f$$.

$head dw$$
The result $icode dw$$ has prototype
$codei%
	%Vector% %dw%
%$$
(see $cref/Vector/reverse_two/Vector/$$ below).
It contains both the derivative
$latex W^{(1)} (x)$$ and the derivative $latex U^{(1)} (x)$$.
The size of $icode dw$$
is equal to $latex n \times 2$$,
where $latex n$$ is the dimension of the
$cref/domain/seq_property/Domain/$$ space for $icode f$$.

$subhead First Order Partials$$
For $latex j = 0 , \ldots , n - 1$$,
$latex \[
dw [ j * 2 + 0 ]
=
\D{ W_0 }{ u_j } \left( x^{(0)} \right)
=
w_0 * \D{ F_0 }{ u_j } \left( x^{(0)} \right)
+ \cdots +
w_{m-1} * \D{ F_{m-1} }{ u_j } \left( x^{(0)} \right)
\] $$
This part of $icode dw$$ contains the same values as are returned
by $cref reverse_one$$.

$subhead Second Order Partials$$
For $latex j = 0 , \ldots , n - 1$$,
$latex \[
dw [ j * 2 + 1 ]
=
\D{ W_1 }{ u_j } \left( x^{(0)} \right)
=
\sum_{\ell=0}^{n-1} x_\ell^{(1)} \left[
w_0 * \DD{ F_0 }{ u_\ell }{ u_j } \left( x^{(0)} \right)
+ \cdots +
w_{m-1} * \DD{ F_{m-1} }{ u_\ell }{ u_j } \left( x^{(0)} \right)
\right]
\] $$


$head Vector$$
The type $icode Vector$$ must be a $cref SimpleVector$$ class with
$cref/elements of type/SimpleVector/Elements of Specified Type/$$
$icode Base$$.
The routine $cref CheckSimpleVector$$ will generate an error message
if this is not the case.


$head Hessian Times Direction$$
Suppose that $latex w$$ is the $th i$$ elementary vector.
It follows that for $latex j = 0, \ldots, n-1$$
$latex \[
\begin{array}{rcl}
dw[ j * 2 + 1 ]
& = &
w_i \sum_{\ell=0}^{n-1}
\DD{F_i}{ u_j }{ u_\ell } \left( x^{(0)} \right) x_\ell^{(1)}
\\
& = &
\left[ F_i^{(2)} \left( x^{(0)} \right) * x^{(1)} \right]_j
\end{array}
\] $$
Thus the vector $latex ( dw[1], dw[3], \ldots , dw[ n * q - 1 ] )$$
is equal to the Hessian of $latex F_i (x)$$ times the direction
$latex x^{(1)}$$.
In the special case where
$latex x^{(1)}$$ is the $th l$$
$cref/elementary vector/glossary/Elementary Vector/$$,
$latex \[
dw[ j * 2 + 1 ] = \DD{ F_i }{ x_j }{ x_\ell } \left( x^{(0)} \right)
\] $$

$head Example$$
$children%
	example/reverse_two.cpp%
	example/hes_times_dir.cpp
%$$
The files
$cref reverse_two.cpp$$
and
$cref hes_times_dir.cpp$$
contain a examples and tests of reverse mode calculations.
They return true if they succeed and false otherwise.

$end
