Updated Review: Thanks to the authors for fixing the plots, the intended message of those plots is now much easier to ascertain. In light of the author response I have decided to keep my review of this work the same, as I believe the authors have successfully demonstrated that the work is of high quality. Originality: The work builds off of a recently published result, the perception-distortion tradeoff, and shows that the original work was incomplete as it did not take into account the third dimension of the tradeoff, classification accuracy. Though the idea of a tradeoff between objectives in signal restoration is not novel, the work is a novel contribution and extension to the theory around this important area of research. Quality: The work is overall of very high quality, with a combination of rigorous proofs, experimental verification, and sound justification for the approach. Clarity: I find the plots in figure 3 to be difficult to easily ascertain the fundamental idea from (though with some effort I understand what they are conveying). I recognize the authors attempt to recreate the plot style from the original perception-distortion paper. Perhaps the authors can find a more intelligible way to break down the many dimensions being displayed in those plots. The paper is otherwise clear and well argued. Significance: The paper strikes me as an extremely significant followup to a recent paper, and relevant to a wide audience at NeurIPS as signal restoration is a topic of high interest, especially in safety critical domains. Better knowledge of these fundamental limitations and trade-offs will help many engineers design for the correct objectives for their applications.

originality: the formal justification of the CDP trade-off is a novel contribution quality: the derivation of the CDP function is technically sound. While inspiring, the empirical justification could be improved: 1) results are only shown on mnist, which is a set of close-to-binary images. How does the CDP tradeoff look like on more general natural images? 2) only one simple degradation scenario of additive Gaussian noise is considered. How does the CDP tradeoff performs beyond this, in the presence other degradations, e.g. blur, low-res, exposure? clarify: the presentation of the paper is clear significance: the general trade-off between classification-distortion-perception is of broad interest to the community of computer vision and machine learning ** Updates ** The authors have partially addressed the concern on generalization beyond simple additive white Gaussian noise degradation, and promise to conduct analysis on datasets beyond mnist. Overall, the formulation and analysis of CDP is a good contribution to the community.

This work is a novel extension of the preceding distortion-perception tradeoff in image restoration tasks. The practicality is enhanced by incorporating analysis of classification errors in related tasks. The claims of this work are technically sound and supported by comprehensive analysis and the experiments. The proof is straightforward and clear. Analysis on the toy example gives solid evidence to support the claim. The overall structure of this paper is clear and easy to follow. Most parts of this work are clearly explained. Issues: - There seems to be an ambiguity between the proposed CDP tradeoff and the discussion in section 4. Specifically, the described CDP suggests that classification error and perceptual naturalness cannot be simultaneously optimized. Yet, in section 4 (line 247-251), the authors also suggested a positive correlation between the classification error rate and "subjective" feeling. This is confusing as it implies the opposite to the original argument. The authors might want to provide a clarification about the statement in section 4 to avoid confusion. - Visual analysis of the experiments can be more thorough. The authors only provided visual results to compare the classification-perception tradeoff with some analysis. While this is not a major concern, it would be more informative if the authors can also provided similar comparisons for distortion and classification. The analysis on visual results can also be more carefully formulated, such that the results can pose a stronger argument. ---------------- I have read the author rebuttal and other reviewers' comments and I tend to recommend this paper. It will be great if the authors can provide more analysis and experimental results beyond MNIST.