NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:6899
Title:Shadowing Properties of Optimization Algorithms

The paper presents a theoretical analysis of how well a discrete dynamic flow approximates the flow/solution of a corresponding ODE for gradient descent and heavy ball methods, e.g., how trajectory of the discrete method with small enough step-size does not deviate too much from the trajectory of the ODE. The main theoretical results are somewhat limited, i.e., small step size and quadratic functinos, but are of interest.