Reviewers find the paper well-motivated and concisely written. While most of the techniques employed in the paper have been investigated in the literature, the work finds a bag of good tricks to solve the phenomenon the authors observed in multi-task batch RL where agents rely on shortcuts to identify tasks and hence do not generalize. Reviewers would like to see more expansion on related works, and better baselines and experiment environment to strengthen the work. Please try to incorporate these feedback when revising your draft.