[2001.02968v1] How to trap a gradient flow
Many more questions remain open on how to exploit the low-dimensional geometry of smooth gradient fields, and the above four questions are only a subset of the fundamental questions that we would like to answer

Abstract We consider the problem of finding an ε-approximate stationary point of a smooth function on a compact domain of Rd. In contrast with dimension-free approaches such as gradient descent, we focus here on the case where d is finite, and potentially small. This viewpoint was explored in 1993 by Vavasis, who proposed an algorithm which, for any fixed finite dimension d, improves upon the O(1/ε2) oracle complexity of gradient descent. For example for d = 2, Vavasis’ approach obtains the complexity O(1/ε). Moreover for d = 2 he also proved a lower bound of Ω(1/ √ ε) for deterministic algorithms (we extend this result to randomized algorithms). Our main contribution is an algorithm, which we call gradient flow trapping (GFT), and the analysis of its oracle complexity. In dimension d = 2, GFT closes the gap with Vavasis’ lower bound (up to a logarithmic factor), as we show that it has complexity O q log(1/ε) ε . In dimension d = 3, we show a complexity of O log(1/ε) ε , improving upon Vavasis’ O 1/ε1.2 . In higher dimensions, GFT has the remarkable property of being a logarithmic parallel depth strategy, in stark contrast with the polynomial depth of gradient descent or Vavasis’ algorithm. In this higher dimensional regime, the total work of GFT improves quadratically upon the only other known polylogarithmic depth strategy for this problem, namely naive grid search.
‹Figure 1: The parallel trap (A parallel trap)Figure 2: The three possible cases for ( e R, e x). e R is marked in red. (A parallel trap)Figure 3: Edge fixing: the rectangle R1 is marked in red. (Edge fixing)