NeurIPS 2020

Improved Analysis of Clipping Algorithms for Non-convex Optimization

Meta Review

Two reviewers indicate acceptance, and one reviewer indicates reject. The main concern of that reviewer is the unrealistic assumption on the bounded noise (or sub-Gaussian tail). Reviewer 1 indicates that this assumption is common and thus acceptable, but also points out it is reasonable to expect that for clipping algorithm the assumption can be relaxed. R1 and R2 pointed out the Lyapunov function seems to be novel, and the analysis is non-trivial. Therefore, I recommend accept. That being said, I agree that this is a valid concern, and suggest the authors to add more discussions on the limitation of this assumption, and also point out that relaxing the assumption for clipping algorithms is more important/urgent than general cases.