NeurIPS 2020

X-CAL: Explicit Calibration for Survival Analysis


Meta Review

For survival analysis, where calibrated models obviously are important, this paper introduces a differentiable plug-and-play regularizer which allows optimizing calibration, and choosing a trade-off between prediction accuracy and calibration. This was considered important and the first of its kind. The paper was intensively discussed among the reviewers. In particular, the reviewers argued whether the paper has shown convincingly enough that the method is necessary, because earlier results indicate other methods may produce calibrated results without the added regularizer (Haider et al. 2018). However, the results the authors point at in their response indicate a positive result, which the authors clarified in their anonymous email. The authors are strongly requested to include the additional results in their paper, as this was the bottleneck issue in recommending acceptance, and to take into account the other important points the reviewers raised. Given the other raised concerns, the paper is still very close to borderline. Finally, I would like to give special thanks to the reviewers who have made an outstanding job in evaluating this paper and taking the feedback from the authors into account. This raises our confidence in the review process which is burdened by the huge volumes at the moment.