NeurIPS 2020

Information theoretic limits of learning a sparse rule

Meta Review

This paper studied the generalized linear model under sublinear sparsity. Theorem 1 establishes reduction of the asymptotic mutual information to a low-dimensional variational problem. Under a certain assumption on the signals, the all-or-nothing phenomenon, previously reported for the linear model under sublinear sparsity, is also shown to be observed in the generalized linear model. The main weakness, as most of the reviewers pointed out, was in a non-rigorous step in the analysis of the all-or-nothing phenomenon, but the authors stated in their response that they have succeeded in providing a rigorous proof on the basis of the suggestion by Reviewer #2 in his review. Now the whole results have been derived rigorously, and all the review scores are well above the acceptance threshold, so that I am glad to recommend acceptance of this paper.