Logarithmic integral optimization via adaptive importance sampling based surrogation methods

Source abstract

Abstract This paper explores Logarithmic Integral Optimization () problems, providing a unified computational framework for various tasks in computational statistics. Key among these are Maximum Likelihood Estimation (MLE) and Maximum a Posteriori (MAP) inference for probabilistic models. Specifically, we investigate scenarios where the model consists of conditional density functions with intractable normalizers. This feature can pose substantial computational challenges for the associated , especially when coupled with the growing prevalence of nonconvex and nondifferentiable modelings in contemporary applications. To address these challenges, we propose an efficient algorithm for , termed Adaptive Importance Sampling-based Surrogation . This method is designed to simultaneously handle nonconvexity and nondifferentiability, while also improving the sampling approximation of the intractable integral term in through variance reduction. The justification of this algorithm is supported by our analysis, which establishes an almost sure subsequential convergence to a necessary candidate for a local minimizer, referred to as a surrogation stationary point . Furthermore, we demonstrate the effectiveness of our algorithm through extensive numerical experiments, confirming its efficiency and stability in facilitating more advanced probabilistic models with intractable normalizers.