NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:6099
Title:Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback

The paper introduces an elegant new two-way compression and the use of momentum for such schemes, which are two important questions and as contributions were clearly appreciated by the reviewers, which reached a good consensus. For related work, we comment that in contrast as mentioned, the parallel case of several workers with EF-SGD is considered already in Section 3.5 of Also, reviewers were asking if experiments could be expanded to clarify more on how much the momentum aspect improves upon EF alone, and the other detailed comments by the reviewers.