This paper generated much discussion amongst the reviewers. The reviewers felt positively about the paper as providing a good combination of deep learning with optimization. The proposed method achieves state-of-the-art performance and reduced computation time, and the experiments are fairly extensive. The paper is generally well written and organized. The biggest weakness of the paper is the minimal technical novelty; reviewers felt that the proposed method is mostly a combination of methods from previous work, with minor modifications. Thus, the reviewers felt that the submission is more of a “systems” paper. There was much discussion about how systems papers should be treated at NeurIPS. Further, as a systems paper, reviewers felt that the paper is missing ablation analyses that can show the importance of each component of their system (e.g. comparison of the correspondences of the proposed system to the correspondences estimated by PWC-Net). As a minor point, reviewers also were interested in seeing more analysis on the performance of the proposed method on larger non-rigid motions, which can be simulated by dropping frames in the test set.