NeurIPS 2020

Model Class Reliance for Random Forests


Meta Review

This is a relevant and timely paper that has been reviewed by four knowledgeable referees, who also thoroughly considered the author's response to their initial reviews. Three of these reviewers recommend acceptance, providing detailed suggestions on how to improve this work before its final submission. R3 recommends rejection. This dissenting opinion was upheld by R3 after discussion with other referees. R3 in my opinion correctly brings up that if the proposed approach aims to improve runtime with an approximate algorithm, this must be sufficiently demonstrated in experiments vs. straightforward alternatives (such as retraining-based methods). That has not been done in the original submission neither in the rebuttal. I find it necessary to include these empirical results, even though the straightforward alternatives feel inferior beforehand. I believe that R3 has a point here - it will be of value to the readers to see exactly how inferior to the proposed approach they might really be. I will recommend marginal acceptance, counting on the authors to follow recommendations and requests from the reviewers in the final copy of their paper.