This work considers the class-imbalance setting, specifically the potential benefits of using semi-supervised/self-supervised learning. Based on theoretical observations regarding unlabeled data in this setting, a pseudo-labeling strategy is proposed for training and pre-training, analyzed, and thoroughly evaluated. Even after rebuttal and discussion, there remained some remaining suggestions around additional citations, etc. (as this is a well-established area), but these were not crucial in my opinion. However, the analysis and empirical findings were considered important by all the reviewers (especially when including the appendices) and there was unanimous support for accepting.