An ODE Inverse Problem Example
Keywords: ode inverse problem example notation forward measurements simulation analytic solution parameter values simulated measurement trapezoidal approximation time grid black box method two levels iteration derivatives simultaneous source

| Prev | Next |  |  |  |  |  |  |

An ODE Inverse Problem Example
Notation
The table below contains the name of a variable, the meaning of the variable value, and the value for this particular example. If the value is not specified in the table below, the corresponding value in ipopt_nlp_ode_problem.hpp can be changed and the example should still run (with no other changes).

Name | Meaning | Value
Na | number of parameters to fit | 3
Ny | number components in ODE | 2
Nz | number of measurements | 4
N ( i ) | number of grid points between `i-1`-th and `i`-th measurement
S ( i ) | number of grid points up to an including the `i`-th measurement

Forward Problem
We consider the following ordinary differential equation: ∂ t y 0 ( t , a ) = - a 1 * y 0 ( t , a ) ∂ t y 1 ( t , a ) = + a 1 * y 0 ( t , a ) - a 2 * y 1 ( t , a ) with the initial conditions y 0 ( 0 , a ) = F ( a ) = ( a 0 0 ) where Na is the number of parameters, a ∈ R Na is an unknown parameter vector. The function and F : R Na → R Ny is defined by the equation above where Ny is the number of components in y ( t , a ) . Our forward problem is stated as follows: Given a ∈ R Na determine the value of y ( t , a ) , for t ∈ R , that solves the initial value problem above.
Measurements
We use Nz to denote the number of measurements. Suppose we are also given a measurement vector z ∈ R Nz and for i = 1 , … , Nz , we model z i by z i = y 1 ( s i , a ) + e i where s i ∈ R is the time for the `i`-th measurement, e i ∼ N ( 0 , σ 2 ) is the corresponding noise, and σ ∈ R + is the corresponding standard deviation.
Simulation Analytic Solution
The following analytic solution to the forward problem is used to simulate a data set: y 0 ( t , a ) = a 0 * exp ( - a 1 * t ) y 1 ( t , a ) = a 0 * a 1 * exp ( - a 2 * t ) - exp ( - a 1 * t ) a 1 - a 2
Simulation Parameter Values

a ¯ 0 = 1 `` | initial value of y 0 ( t , a )
a ¯ 1 = 2 `` | transfer rate from compartment zero to compartment one
a ¯ 2 = 1 `` | transfer rate from compartment one to outside world
σ = 0 `` | standard deviation of measurement noise
e i = 0 `` | simulated measurement noise, i = 1 , … , Nz
s i = i * .5 `` | time corresponding to the `i`-th measurement, i = 1 , … , Nz

Simulated Measurement Values
The simulated measurement values are given by the equation z i = e i + y 1 ( s i , a ¯ ) = e i + a ¯ 0 * a ¯ 1 * exp ( - a ¯ 2 * s i ) - exp ( - a ¯ 1 * s i ) a ¯ 1 - a ¯ 2 for k = 1 , … , Nz .
Inverse Problem
The maximum likelihood estimate for a given z solves the following inverse problem minimize ∑ i = 1 Nz H i [ y ( s i , a ) , a ] w . r . t a ∈ R Na where the functions H i : R Ny × R Na → R is defined by H i ( y , a ) = ( z i - y 1 ) 2
Trapezoidal Approximation
This example uses a trapezoidal approximation to solve the ODE. This approximation procedures starts with y 0 = y ( 0 , a ) = ( a 0 0 ) Given a time grid { t i } and an approximate value y i -1 for y ( t i -1 , a ) , the a trapezoidal method approximates y ( t i , a ) (denoted by y i ) by solving the equation y i = y i -1 + [G ( y i , a ) + G ( y i -1 , a ) ] * t i - t i -1 2 where G : R Ny × R Na → R Ny is the function representing this ODE; i.e. G ( y , a ) = ( - a 1 * y 0 + a 1 * y 0 - a 2 * y 1 ) This G ( y , a ) is linear with respect to y , hence the implicit equation defining y i can be solved inverting the a set of linear equations. In the general case, where G ( y , a ) is non-linear with respect to y , an iterative procedure is used to calculate y i from y i -1 .
Trapezoidal Time Grid
The discrete time grid, used for the trapezoidal approximation, is denoted by { t i } which is defined by: t 0 = 0 and for i = 1 , … , Nz and for j = 1 , … , N ( i ) , Δ t i = ( s i - s i -1 ) / N ( i ) t S ( i -1 ) + j = s i -1 + Δ t i * j where s 0 = 0 , N ( i ) is the number of time grid points between s i -1 and s i , S ( 0 ) = 0 , and S ( i ) = N ( 1 ) + … + N ( i ) . Note that for i = 0 , … , S ( Nz ) , y i denotes our approximation for y ( t i , a ) and t S ( i ) is equal to s i .
Black Box Method
A common approach to an inverse problem is to treat the forward problem as a black box (that we do not look inside of or try to understand). In this approach, for each value of the parameter vector a one uses the trapezoidal approximation (on a finer grid that { s i } ) to solve for y 1 ( s i , a ) for i = 1 , … , Nz .
Two levels of Iteration
As noted above, the trapezoidal approximation often requires an iterative procedure. Thus, in this approach, there are two levels of iterations, one with respect to the parameter values during the minimization and the other for solving the trapezoidal approximation equation.
Derivatives
In addition, in the black box approach, differentiating the ODE solution often involves differentiating an iterative procedure. Direct application of AD to compute these derivatives requires a huge amount of memory and calculations to differentiate the iterative procedure. (There are special techniques for applying AD to the solutions of iterative procedures, but that is outside the scope of this presentation).
Simultaneous Method
The simultaneous forward and inverse method uses constraints to include the solution of the forward problem in the inverse problem. To be specific for our example, minimize ∑ i = 1 Nz H i ( y N ( i ) , a ) w . r . t y 1 ∈ R Ny , … , y S ( Nz ) ∈ R Ny , a ∈ R Na subject to y j = y j -1 + [G ( y j -1 , a ) + G ( y j , a ) ] * t j - t j -1 2 for j = 1 , … , S ( Nz ) y 0 = F ( a ) where for i = 1 , … , Nz , N ( i ) is the number of time intervals between s i -1 and s i (with s 0 = 0 ) and S ( i ) = N ( 1 ) + … + N ( i ) . Note that, in this form, the iterations of the optimization procedure also solve the forward problem equations. In addition, the functions that need to be differentiated do not involve an iterative procedure.
Source
The file ipopt_nlp_ode_problem.hpp contains source code that defines the example values and functions defined above.

Input File: cppad_ipopt/example/ode2.omh