NeurIPS 2020

A shooting formulation of deep learning

Meta Review

The paper presents a new perspective on parameterizing and training continuous-depth neural networks from the vantage point of control theory. The paper can be understood as an extension of the neural-ODE line of work. In particular they consider a particle based parameterization of a neural-ODE that can be solved for via a shooting method. All reviewers agree that this is an innovative, new, approach that is theoretically sound and that the paper is well written with good coverage of the shooting method (for the unfamiliar reader). The provided implementation further aids reproducibility. The only small weakness is that the experiments in the paper are largely on small-scale problems, and it is thus not completely obvious whether the presented approach will yield improvements over NODE in other scenarios. Overall this is a good paper that is well motivated, well written and makes an interesting theoretical connection. It should clearly be accepted.