This is a mixed paper. In terms of problem setting and methodology, the proposed problem is interesting, and the proposed method is novel. In terms of experiments, there are some concerns on the experimental setup. After rebuttal, two reviewers still have concerns on the practical issues of the proposed method on real-world scenarios. One is not convinced by authors' response on how to apply the proposed method to handle the special case, where the anomalies are partially overlapped with the normal patterns. The other is not satisfied that the authors do not conduct additional experiments on larger CV datasets, who even lowers the overall score from 6 to 4. After reading through this paper, I find that the proposed idea is interesting and novel, which does provide some new insights to the field of transfer learning in the application to abnormal detection. I believe this is more important and useful than achieving SOTA results by developing incremental methods. Though there may be some practical issues when applying the proposed method to real-world complex scenarios as mentioned by reviewers, the proposed method looks reasonable to most general cases of cross-domain anomaly detection. Of course, if the authors can address the issues raised by the reviewers, e.g., conducting more experiments on larger CV datasets (though I do not think it is a must), and address the overlapping issue of anomalies and normal patterns, then they can make this work stronger. Overall, by considering the technical novelty, I think this work is still deserved to be published in NeurIPS.