NeurIPS 2020

Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessian

Meta Review

This paper proposes Ridge Regression, a novel method for exploring local minima using eigenvectors of Hessian. The novelty of the method was appreciated and it was also noted that the method can connect to classic methods such as Branch and Bound technique. There was also appreciation of the empirical results. However, the lack of mathematical insights related to diversity of solutions, --a key claim of the paper-- was missed. Even if there was no analysis a comparison with the random orthonormal set of descent directions in producing diverse directions would have been illustrative to understand the power of the method. The overall consensus is that the paper while having excellent novelty lacks mathematical rigour and thus did not have any strong advocate.