HENG Jeremy, XU Yaxian, BISHOP Adrian N.
We consider a class of finite-time horizon nonlinear stochastic optimal control problem. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging task. We propose a new Monte Carlo approach that significantly improves upon existing methodology. We tackle the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state-space model, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce the cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with O(ϵ2) mean squared error with a cost of O(ϵ−2log(ϵ)2). In contrast, a cost of O(ϵ−3) is required for the existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples.
JASRA, A., HENG, J., XU, Y. et BISHOP, A.N. (2022). A Multilevel Approach for Stochastic Nonlinear Optimal Control. International Journal of Control, 95(5), pp. 1290-1304.