I've read all reviews and author response and I would like to recommend acceptance. However, I would like to ask the authors to: 1) introduce some analysis, which most of the reviewers are asking for and was also promised in the author response 2) provide some comparison to the very recenlty introduced Gomez et al 2020 dataset. It is definately OK to have more datasets for the same topic (if any, the more the better), but I would say it's quite important to also have a feeling of what are there differences (minimally, on a high-level). R1/R2/R4 all think this is a very interesting contribution that can spur more work on detecting and dealing with hateful speech on social media. Moreover, reviewers agree that this dataset was constructed carefully so as to disadvantage uni-modal models, thus providing a good test bed for multimodal reasoning. R3 has expressed a number of concerns with regards to organization of the paper, use of simple baselines but also lack of analysis. While I disagree with the first 2 points, none of these points warrant rejection of the paper.