NeurIPS 2020

Improving Generalization in Reinforcement Learning with Mixture Regularization

Meta Review

This submission was generally understood by reviewers to be a straightforward extension of existing work on supervised learning regularization, thus presenting limited technical novelty. It was reasonably well executed from an experimental perspective and potentially high impact given the strength of the results. In discussion, reviewers debated the merits of the paper, with several arguing that for such a limited algorithmic contribution the analysis component needed to be stronger. R3 would have liked to see broader empirical assessment, a greater discussion and interrogation of limitations, and whether combination with other forms of data augmentation yielded additive gains, while R1 felt that evaluation on strictly image-based environments was potentially misleading. I concur with several of these criticisms, but must balance the paper's shortcomings with the value to the community in highlighting a method which is a very clear target for further research, and an already potentially useful entry in a practitioner's toolbox.