The paper proposes a way to incorporate weak supervision, in the form of pairwise comparisons along various axes, into a goal-directed reinforcement learning framework, showing how this supervision can identify relevant latent factors for the construction of new tasks. The reviewers agree that this is a novel approach and makes an important step toward fully unsupervised approaches. As such, we are recommending acceptance.