分类： Delphi
2012-02-16 10:50:59
The Hamilton–Jacobi–Bellman (HJB) equation is a which is central to theory. Classical variational problems, for example, the can be solved using this method. The HJB method can be generalized to systems as well.
The solution of the HJB equation is the 'value function', which gives the optimal cost-to-go for a given with an associated cost function. The solution is open loop, but it also permits the solution of the closed loop problem.
The equation is a result of the theory of which was pioneered in the 1950s by and coworkers. The corresponding discrete-time equation is usually referred to as the . In continuous time, the result can be seen as an extension of earlier work in on the by and .
|
Contents[]
|
Consider the following problem in deterministic optimal control
where C[] is the scalar cost rate function and D[] is a function that gives the economic value or utility at the final state, x(t) is the system state vector, x(0) is assumed given, and u(t) for 0 ≤ t ≤ T is the control vector that we are trying to find.
The system must also be subject to
where F[] gives the vector determining physical evolution of the state vector over time.
[] The partial differential equationFor this simple system, the Hamilton Jacobi Bellman partial differential equation is
subject to the terminal condition
where the means the of the vectors a and b and
is the operator.
The unknown scalar V(x,t) in the above PDE is the Bellman '', which represents the cost incurred from starting in state x at time t and controlling the system optimally from then until time T.
[] Deriving the equationIntuitively HJB can be "derived" as follows. If V(x(t),t) is the optimal cost-to-go function (also called the 'value function'), then by Richard Bellman's , going from time tto t + dt, we have
Note that the of the last term is
where o(dt2) denotes the terms in the Taylor expansion of higher order than one. Then if we cancelV(x(t), t) on both sides, divide by dt, and take the limit as dt approaches zero, we obtain the HJB equation defined above.
[] Solving the equationThe HJB equation is usually , starting from t = T and ending at t = 0.
The HJB equation is a for an optimum. If we can solve for Vthen we can find from it a control u that achieves the minimum cost.
In general case, the HJB equation does not have a classical (smooth) solution. Several notions of generalized solutions have been developed to cover such situations, including ( and ), (), and others.
[] Extension to stochastic problemsThe idea of solving a control problem by applying Bellman's principle of optimality and then working out backwards in time an optimizing strategy can be generalized to stochastic control problems. Consider similar as above
of the latter does not necessarily solve the primal problem, it is a candidate only and a further verifying argument is required. This technique is widely used in Financial Mathematics to determine optimal investment strategies in the market (see for example ).
[] See also