This paper received two positive and one negative reviews, and the negative one was very short and non-specific, so normally it would be an accept (and so I will ultimately recommend). A weakness of the paper not noted by the reviewers is that the authors are apparently unaware of the related paper by Travnik et al. (see below). The Travnik paper subtracts slightly from the novelty of the current work but adds to the recognition of the importance of the real-time issues. On balance, I don’t think that the existence of this prior work diminishes the case for publication of this paper. (However, I do think the paper should be revised to refine its claim to novelty and to cite the Travnik paper). reference: Travnik, J. B., Mathewson, K. W., Sutton, R. S., & Pilarski, P. M. (2018). Reactive Reinforcement Learning in Asynchronous Environments. Frontiers in Robotics and AI, 5, 79.