All four reviewers unanimously voted for acceptance. They appreciated that the paper addresses an important and timely question (When is self-training beneficial over pre-training?) and numerous experiments and analyses provide novel insights that will encourage future research in this direction. The initial reviews pointed out a few (minor) concerns and the rebuttal adequately addressed them. In response, R2 increased the rating from 6 to 7 after the discussion phase. Overall, I think this is a solid paper and would love to see it at the conference.