NeurIPS 2020

An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay

Meta Review

The paper provides some theoretical treatment of prioritized experience replay, and shows how the weighted sampling scheme can be viewed as minimizing a different loss function under the uniform sampling scheme. The main insight here is that since the weights are derived from the loss itself, there is some cancellation that changes the original loss function. The insights are used to derive two new algorithms which perform reasonably well in experiments. The paper is interesting, uses new theoretical insights to derive algorithms with competitive performance. As such, we recommend acceptance.