Separation Laws for Flip Bifurcations and Endogenous Switching Boundaries in Gradient Maps

Source abstract

Gradient clipping is widely used to control large updates in optimization, while finite-step gradient descent develops period-two oscillations near the edge of linear stability. Recent work has analysed these oscillations via smooth flip bifurcation theory and higher-order stability criteria, whereas piecewise-smooth bifurcation theory describes generic interactions between period-doubling and switching boundaries. It remains unclear how a smooth gradient-descent oscillation first interacts with the switching surface generated endogenously by radial gradient clipping, and what dynamical structure emerges after this contact. For the finite-dimensional clipped gradient map $F_{\eta,c}(x) = x - \eta \, C_c(\nabla V(x))$, we analyse this transition near a nondegenerate strict local minimum with a simple maximal Hessian eigenvalue. Clipping is locally inactive near the minimum, so the fixed-point stability threshold and the local flip bifurcation coincide with those of ordinary gradient descent. Under a supercritical flip condition, the stable period-two branch reaches the clipping boundary at $\eta_{\mathrm{sc}}(c) = \eta_f + K c^2 + O(c^4)$ with $\eta_f = 2/\lambda_*$, where $K = \Gamma/(3\lambda_*^4 \|v_*\|^2)$ is determined explicitly by derivatives of the potential up to fourth order and by the clipping norm. Beyond the separation law, the gradient structure yields rigidity absent from generic switching maps: the two points of every smooth period-two orbit have opposite gradients and hence reach a radial clipping boundary simultaneously; mixed smooth--clipped period-two itineraries with one strictly smooth and one strictly clipped point are impossible; and every strictly fully clipped period-two orbit has an exact unit Floquet multiplier and is therefore nonhyperbolic. A solvable planar model realizes the full transition from a stable smooth two-cycle to a neutral family of fully clipped period-two cycles.