NeurIPS 2020

Stochastic Optimization with Laggard Data Pipelines

Meta Review

The paper is a theoretical analysis of the behaviour of "echoed gradients" in convex optimization. The investigation is timely, and will cast light on an interesting area of current practice. More than one reviewer believes the paper should explicitly handle the non-convex case. I disagree, and side with the authors that the convex case is sufficient. The relevant non-convex optimizers generally contain convex stepping as a subprogram, so this analysis is reasonable.  A point where I do side strongly with the reviewers is the topic of carelessness in presentation. All agree the paper is well *written*, but that there are errors in the math. The authors rebuttal dismisses these complaints as mere "typos" or "nuance". I find this rather unconvincing. Typos in math are considerably harder on the reader than typos in prose, and the tone of the rebuttal seemed to imply it was the reviewers' fault for not autocorrecting the typos rather than the authors' for including them.