All reviewers are positive about the paper. The paper introduces a new initialization scheme for ResNets. The experimental results the authors present are extensive. The proposed initialization scheme appears to be quite effective empirically. The discussion of the results is particularly careful and nuanced. The contributions are of broad interest to the machine learning community. We recommend to take the reviewers' comments and suggestions into account while preparing the camera ready final version of the paper. Accept.