Jul 23, 2019 · In this paper, we answer a question: can the nonconvex heavy-ball algorithms with random initialization avoid saddle points? The answer is yes!
Theoretically, we prove that heavy-ball gradient descent enjoys larger stepsize than the gradient descent to escape saddle points to escape the saddle point.
Theoretically, it is proved that heavy-ball gradient descent enjoys larger stepsize than the gradient descent to escape saddle points to escape the saddle ...
Theoretically, we prove that heavy-ball gradient descent enjoys larger stepsize than the gradient descent to escape saddle points to escape the saddle point.
People also ask
What is the heavy ball algorithm?
What is a saddle point in the context of gradient descent?
In this paper, we answer a question: can the nonconvex heavy-ball algorithms with random initialization avoid saddle points? The answer is yes! Direct using the ...
Theoretically, we prove that heavy-ball gradient descent enjoys larger stepsize than the gradient descent to escape saddle points to escape the saddle point.
The easiest way to keep your publications page up to date.
This paper develops a new perturbation mechanism for optimization algorithms. In this mechanism, perturbations are adapted to the history of states via the ...
Heavy ball method: The heavy ball method was originally proposed by Polyak (1964). It has been observed that this algorithm, even in the deterministic setting, ...
Jun 4, 2024 · I.: Accelerated gradient descent escapes saddle points faster than gradient descent. In: S. Bubeck, V. Perchet, and P. Rigollet (eds ...