Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
The paper is very well written, however, there are several questions about novelty of the work detailed below
- Originality To the best of my knowledge, this is a novel work. The representational power of neural ODE models has not been studied much in the field. Although the negative examples and proof techniques are standard results in point-set topology and metric spaces, the appropriate application makes the idea very interesting. The related works section seems adequate. - Quality The work is technically sound, with both cleanly written proofs and comprehensive empirical analysis. - Clarity The work is clear, concise, and coherent. - Significance See part 1 of the review.
Originality: The method is original in the deep learning literature. Though limitations of ODEs cannot cross paths is quite well-known, this paper views this deficiency from a modeling perspective and removes it while keeping within the ODE framework. Quality & Clarity: The motivations for ANODE are well-explained and the experiments are well-chosen. The prose is very well written, and with many simple visualizations that support their claims. Significance: Given the interest in ODE-based modeling, this work has enough impact for a NeurIPS paper. Comments: - I think reporting _only_ cross entropy loss for image classification tasks is a bit weird. If it makes sense to compare cross-entropy because it is objective that is being minimized for training (e.g. training instability plots), sure. But I think showing classification accuracy would be more meaningful and allow more follow-up works as it is the metric of interest. It is often the case in image classification that while validation cross entropy increases, the classification actually gets better. Right now, Figure 11 makes it seem like ANODE has a more significant overfitting problem, though I think the accuracy probably shouldn't increase very much even if the loss increases.