Stochastic Zeroth-Order Multi-Gradient Algorithm for Multi-Objective Optimization
Zhihao Li, Qingtao Wu, Moli Zhang, Lin Wang, Youming Ge, Guoyong Wang
Source abstract
Multi-objective optimization (MOO) has become an important method in machine learning, which involves solving multiple competing objective problems simultaneously. Nowadays, many MOO algorithms assume that gradient information is easily available and use this information to optimize functions. However, when encountering situations where gradients are not available, such as black-box functions or non-differentiable functions, these algorithms become ineffective. In this paper, we propose a zeroth-order MOO algorithm named SZMG (stochastic zeroth-order multi-gradient algorithm), which approximates the gradient of functions by finite difference methods. Meanwhile, to avoid conflicting gradients between functions and reduce stochastic multi-gradient direction bias caused by stochastic gradients, an SGD-type method is adopted to acquire weight parameters. Under the non-convex setting and mild assumptions, the convergence rate is established for the SZMG algorithm. Simulation results demonstrate the effectiveness of the SZMG algorithm.
Evidence graph
No public relationships recorded yet.
Integrity note: This page is a factual metadata record created by deterministic ingestion. It is not a claim that the work moves a mathematical frontier or has been independently verified.