This paper received mixed reviews. All reviewers found the empirical findings in the paper to be very interesting. The main concern from reviewers was about the lack of theoretical justification for the findings. However, many empirical results precede theoretical results, and this paper's empirical results are interesting in its own right. The area chair has read the paper in detail. The paper is well written, and provides important empirical analysis for two timely questions in the field today: model robustness and self-supervised learning. The paper's many experiments on accuracy, out-of-distribution detection, and robustness make this work potentially very interesting to the research community. Although the techniques in the paper are straightforward, the empirical results establish a novel (to AC and reviewers knowledge) link between self-supervised learning and model robustness. The AC recommends acceptance.