NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:2624
Title:When does label smoothing help?

In this paper, the authors provide a comprehensive empirical study on label smoothing for deep learning. The study contains quite a few interesting insights, e.g., label smoothing implicitly calibrates learned models by making their predictions more consistent with the underlying accuracy; while teacher networks trained with label smoothing get more accurate, they are ultimately less effective at distilling knowledge into a student network. The reviewers raised some concerns, including the relatively narrow scope of the experiments, and the lack of theoretical analysis. The authors did a good job in rebuttal, and all the reviewers agree after reading the rebuttal that the empirical insights in this paper have high practical value, and NeurIPS audience should benefit from them