NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:583
Title:A Condition Number for Joint Optimization of Cycle-Consistent Networks

Reviewer 1

Summary: Overall the paper is strong in originality, quality, clarity, and significance. It seems like a strong accept. Maybe the weakest area is significance, because while a clear improvement over existing methods, the tasks are still fairly restricted (i.e. rigid body dense flow, correspondence within a collection of similar shapes), and the results are essentially incremental. All comments raised by other reviewers and responses were similarly minor, so there is no change in the review between the pre- and post-rebuttal phases. Originality: -The paper has some similarities to Zhang et al. '19 [45]. But overall it seems like there is easily sufficient novelty compared to existing work because of both theoretical and experimental justification for cycle-consistent bases compared to path-invariant bases. Quality: The experimental results section is thorough with good metrics and convincing results. It could have been better with some figures containing more detailed qualitative results. This could aid in making the experimental setups easier to understand and would improve result interpretability. -A limitations section would have been nice. Understanding the cases that are hardest for method could be useful for future research. Clarity: This paper is exceptionally well written. It sets up a clear problem statement, motivates it, and explains the challenges and key idea, all in an easy to parse way. It also does an excellent job at keeping the main paper clean and relatively reasonable to get through. It does so in part by hiding most of the proof complexity in the supplemental and using informal versions of those theorems in the main paper. Please note that the formal proofs in pages 12-21 of the supplemental are quite complex and extensive-- they were referenced while reviewing the paper but not validated. -The ~ operator on edges is never defined. Presumably it means path concatenation. The notation is also a bit loose with the distinction between paths and cycles. For instance, Remark 3 seems a bit premature since the paper hasn't actually defined the notion of a path invariance basis. Reading [45] is currently necessary to understand parts of this paper. -Minor grammatical error on line 219 (stand-alone dependent clause). Significance: -It would be nice to show this form of cycle-consistency optimization enables novel capabilities, rather than just improved quantitative results over existing methods. -"We always assume f_{ij} is an isomorphism between D_i and D_j and f_{ji} = f^{-1}_{ij}" Is this really an okay assumption in practice? Is anything given up by this assumption? Most parametric maps will not be injective.

Reviewer 2

Originality: This paper extends the cycle-basis [22] to the cycle-consistency basis, more specifically, defining the condition number of cycle-consistency basis and minimizing the condition number. Both the problem and the algorithm are novel. Quality & clarity: Theoretically, the paper provides thorough definitions/derivations for cycle consistency bases. It also clarifies the differences with path-invariance bases on directed graphs and draws connections to cycle bases. Technically, the paper jointly optimizes the objective via the 3-step method. Experiments are conducted in a principled manner. In general, the paper present high quality in the three aspects. The paper is also well written. Significance: Since cycle-consistency is very commonly used in computer vision and the other areas, the paper has the potential to benefit all the related algorithms. Post rebuttal: The rebuttal has addressed my concern w.r.t the scales of the networks, the possible overfitting problem, and the testing strategy. I thus would like to keep the original rating.

Reviewer 3

The paper first introduces the concept of cycle-consistency basis, and then presents a three-step algorithm that effectively samples a subset of weighted cycles to enforce the cycle-consistency constraint for optimizing maps among a collection of related objects/domains. Some novel techniques such as condition number are applied to compute the cycle-consistency basis. It is hypothesized that the proposed approach for cycle selection is superior to others in that it takes the convergence behavior of the induced joint optimization problem into account. The experiments on two settings, including consistent shape correspondences and consistent neural networks among multiple domains, demonstrate the strength of the algorithm in map quality. Pros: - The applied methods are straightforward and appropriate. - The proposed condition-number-based technique for stability scoring is well motivated. - On a variety of settings, the approach outperforms state-of-the-art techniques. And it is suitable for diverse map representations. Hence, the present work may exhibit broad impact. - Detailed mathematical proofs are provided for every proposition and theorem, which makes it theoretically sound. - The paper is overall well written (with a few caveats detailed below). Cons/comments: - The algorithm needs to be summarized more explicitly. I would prefer adding some pseudocode for better explanation. - More details should be included in experiments. For example, the hyperparameters in line 217 are unexplained, and necessary comparison in terms of running time is missing. - There are some ambiguous expressions, such as ā€œsā€ in Theorem 4.1. - It is recommended to change the title so that highlights of the work could be well reflected. - It would be better to give a brief introduction to path-invariance bases before Remark 3. - Some grammatical mistakes (line 8, line 219).