NeurIPS 2020

Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts

Meta Review

This paper focuses on the use of "citizen science" to train large neural networks. An algorithm is proposed that is fault-tolerant to missing/slow/unreliable nodes, and some preliminary experiments are carried out to demonstrate its utility. The reviewers initially suggested that the experiments were limited, but after rebuttal were convinced that the paper is worth publishing in its current form.