NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:2742
Title:Breaking the Glass Ceiling for Embedding-Based Classifiers for Large Output Spaces

There is some disagreement about the significance of the paper among the reviewers. Three steps can be distinguished. First, to refute the common belief that low-dimensional embeddings act as bottlenecks that limit the accuracy in the extreme classification case. Here, while it is true (raised by reviewer 1) that a representation result does not imply computational achievability, I feel that it reverses the direction of justification. If someone could show that common optimization methods fail to find embeddings (which "exist"), then this would re-instantiate the argument, yet in a more refined/precise form. I still feel that the paper makes an important argument in an ongoing discussion. Second, the finding that overfitting is the key problem. I agree here with reviewer 2 that overfitting has been observed in many papers before, but I feel the current submission makes a more focussed claim and it proposes a novel regularizer to address it in a specific manner. All in all, while the paper can certainly be improved in many ways, I feel it does enrich and advance our understanding of embeddings based methods for extreme classification,