This paper is extremely borderline, and the reviewers were split during the discussion. The video classification experiment in the appendix is quite nice and is critical for illustrating the generality of the method beyond image classification. However, these results appear somewhat preliminary with no comparisons to unsupervised learning methods, and should be highlighted in the main text of the paper, rather than coming across as an after thought. Further examples of results in other domains would further strengthen the contribution of this paper to the general ML community. As it stands, we think that the contributions of the paper are valuable to the NeurIPS community. In the final version, we strongly encourage the authors to include a more thorough discussion of the broader method, and include results in the main paper that better exemplify and analyze the generality of the method.