Asymptotic analysis of a dynamical system with Hessian-driven damping
for smooth non-convex optimization problem
Abstract
In this paper, we concentrate on the non-convex optimization (nc-PL)
problem under Polyak-Šojasiewicz condition. Most existing non-convex
optimization algorithms for nc-PL utilize the dynamical systems to
observe the iterative behavior trajectories. However, the dynamical
systems often yield oscillations due to its undamped form, which leads
to slow convergence or divergence. To solve this problem, we derive a
second-order continuous dynamical system with Hessian-driven damping,
which can avoid oscillations. By Cauchy-Lipschitz-Picard theorem, we
first derive the existence and uniqueness of global solution for this
dynamical system. Further, we present the stability of small
perturbation around the global solution and the asymptotic convergence
of the objective function along the trajectories generated by the
dynamical system for solving nc-PL without assuming uniqueness of the
minimizer. Moreover, we show the linear convergence rate of the
dynamical system for smooth convex optimization. Finally, by replacing
the objective function f by its Moreau envelope, we extend the
results to the case of non-smooth convex functions.