NeurIPS 2020

Neural Networks Learning and Memorization with (almost) no Over-Parameterization

Meta Review

This paper studies optimization in the NTK regime, further improving the best prior width bounds for random data (I believe Oymak-Soltanolkotabi were the prior best). The reviewers and I were all favorable, and I look forward to seeing this paper appear, and support the authors in further investigations. --- Minor comments. (a) A reviewer pointed out that Lemma 12 did not receive a full proof: please provide a complete proof in your revisions. Relatedly, this point was not sufficiently handled in the rebuttal, despite the rebuttal using less than half a page. Please consider such things in the future. (b) I believe there are some newer works now which should be cited? One is by Roman Vershynin, and I believe Sebastien Bubeck and colleagues also had a paper on the "Baum" problem. (c) The title of the paper does not match between tex and CMT?!