NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:5830
Title:How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

All reviewers are positive about the paper. The paper introduces a new initialization scheme for ResNets. The experimental results the authors present are extensive. The proposed initialization scheme appears to be quite effective empirically. The discussion of the results is particularly careful and nuanced. The contributions are of broad interest to the machine learning community. We recommend to take the reviewers' comments and suggestions into account while preparing the camera ready final version of the paper. Accept.