Duality Theory for Filter Stability in Hidden Markov Models

Source abstract

This article is concerned with the problem of nonlinear (stochastic) filter stability for a discrete time hidden Markov model (HMM) with finite state and observation spaces. For this purpose, a comprehensive duality theory is developed whereby the dual control system is a backward stochastic difference equation (BSDE). The duality is expressed in two inter-related forms. In its variational form, the minimum mean squared error of optimal filtering equals the value of a dual optimal control problem. In its structural form, controllability (resp. stabilizability) of the BSDE and observability (resp. detectability) of the HMM are dual properties. The dual optimal control problem supplies the Lyapunov function and the dissipation: the former is the value function (a variance), the latter the optimal running cost accumulated over the horizon. The main result is an invariance principle: under any one of three model properties---ergodicity of the Markov matrix, or observability or detectability of the HMM---zero dissipation over a finite horizon forces the Lyapunov function to vanish. A converse is also obtained: detectability is necessary. Two applications are given. The first is filter stability in chi-squared divergence: the invariance principle is applied in the manner of LaSalle, to limits of trajectories along which the dissipation vanishes asymptotically, the role of the model properties being to render the zero-dissipation set trivial. The second is a quantitative form of the invariance principle, through a Poincare constant, from which convergence rates follow. Both the model properties that govern stability and the Lyapunov function that certifies it are thus obtained from a dual optimal control problem, as in the classical theory for the linear Gaussian model.