Return to results
Journal articles (2020), International Journal of Control, In press

A Multilevel Approach for Stochastic Nonlinear Optimal Control

JASRA Ajay , HENG Jeremy, XU Yaxian, BISHOP Adrian N.

We consider a class of finite-time horizon nonlinear stochastic optimal control problem. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging task. We propose a new Monte Carlo approach that significantly improves upon existing methodology. We tackle the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state-space model, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce the cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with O(ϵ2) mean squared error with a cost of O(ϵ−2log(ϵ)2). In contrast, a cost of O(ϵ−3) is required for the existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples. Link to the article

JASRA, A., HENG, J., XU, Y. and BISHOP, A.N. (2020). A Multilevel Approach for Stochastic Nonlinear Optimal Control. International Journal of Control, In press.

Keywords : #optimal-control, #multilevel-Monte-Carlo, #Markov-chain-Monte-Carlo, #sequential--Monte-Carlo