The reviewers agreed that this is a nice paper that takes a highly novel approach to the important problem of off-policy evaluation. The main concern is how restrictive the problem setting is, but this is at least somewhat alleviated by the discussion and additional results in the author's rebuttal. The authors are strongly encouraged to integrate this into the final version. There were also minor concerns about readability and the lack of structure in the paper, so the authors are also strongly encouraged to significantly improve this in the final version.