NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:2198
Title:Uncoupled Regression from Pairwise Comparison Data

Reviewer 1

This paper proposes two novel approaches for the uncoupled regression problem, in which the correspondance between the input data and the targets is not known. Instead these methods use unlabeled data and pairwise comparisons of the targets. The first approach is the risk approximation approach (RA) and it minimizes an approximation of the risk defined based on the expected Bregman divergence. The second approach, called the target transformation (TT) approach, consists in mapping the target variable to a uniformly distributed random variable using the cumulative distribution function. Estimation error bounds are derived for each method. The empirical performances of the proposed methods are evaluated on synthetic data and on benchmark datasets. The paper is clearly written and well organized. To my knowledge, the proposed approach is novel. I think that the contributions of this paper are significant from a theoretical and also from an empirical point of view. Indeed the obtained results are convincing as they show that with many pairwise comparison data the proposed methods obtain close performances to supervised linear regression. I have a remark on the line 276: the authors claim that the TT approach outperforms the RA approach with sufficient pairwise comparison data however Figure 1 shows that the RA approach always obtains a smaller MSE than TT. Maybe there was some confusion with Figure 2 ? In addition the error bars on Figures 1 and 2 seem a bit stange. Minor comments: there is a typo on the line 255 (squared). Also I think that it should be the derivative of phi in the first term of Equation 7 and not phi. %%% Update: I have read the answers from the authors and I thank them for addressing some of my concerns, especially regarding the addition of other uncoupled regression methods in the comparison.

Reviewer 2

Authors propose a theoretically sound solution to uncoupled regression via pairwise ranking. They utilize statistical learning theory. They also analyze the estimators properties such as variance and develop generalization bounds. Furthermore they also supply empirical evidence for their algorithm. Paper is well written and connected to literature. Experimental study is well connected. My concerns are the practical value of the contribution and computational requirements,

Reviewer 3

The paper proposes two novel methods for uncoupled regression by employing pairwise comparison data within the framework of empirical risk minimization. Compared to existing methods, the new approaches can get rid of the strong assumptions on the target function by adopting the ideas of risk approximation or target transformation. The authors derive the estimation error bounds and demonstrate by numerical experiments that the proposed methods perform comparably to the ordinary supervised regression. The results are novel and very interesting. The paper is well written and easy to follow.