The reviewers and I found this paper to be well-motivated and the question of how the spectrum of the covariance relates to robustness is one that the community would surely be interested in. There was some discussion regarding the methodology employed to investigate this question. First, there was general agreement that experiments on natural images would significantly improve the paper, especially because the spectrum of the input would be materially different. I do not believe that it is appropriate to relegate this experiment to future work, and I am not convinced by the rebuttal that including experiments with natural images would be computationally prohibitive. For example, downscaling or subsampling from CIFAR-10 would be a realistic option if the computations were too onerous on the full dataset. While I do not regard this analysis as mandatory for acceptance, I would strongly encourage the authors to include it in a revision. Second, there is some arbitrariness in the specifics of the regularization method and how it was implemented. More focus should be given to some of the details and how they do/do not affect the results. I am specifically concerned about the batch size and the effect of freezing the eigenvectors. Even some empirical justification for the chosen method/configuration would go a long way. Again, this analysis is not mandatory for acceptance, but including it would significantly enhance the takeaways that readers could draw from this paper. Overall, this is a borderline paper, but I think it just crosses the bar, and would be a strong paper if the authors included the above recommendations into the final version. So I recommend acceptance.