NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:4461
Title:Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model

The paper analysis momentum SGD on the noisy quadratic model and provides empirical results looking at varying batch size, momentum, ane preconditioning. Well written paper. Also, reviewers had several suggestions but saw the insight that the idealized model provides to be useful.