I think the paper was very insightful and represents a significant advance. It was well written and well explained. It combines two powerful ideas, using feedback from a small validation set, and a fresh theoretical perspective linking RL ideas to the gradient descent realm. ------------------------------------------- Post Rebuttal --------------------------------------- I've read the author rebuttal and other reviews. The authors seems to have replied to issues other reviewers had in appreciating the novelty/significance. Myself, I still honestly think that if I randomly sampled 19 other accepted NIPS papers and this one, it would be the most significant and interesting read. I'm comfortable remaining an isolated outlier for this evaluation.

The paper propose a system that takes advantage of combining an off-the-shelf RL model with supervised training to create different parameterizations of "data reward", which helps guide the weighting and augmentation of training data. The method is clearly described; however, given the idea of RL for data weighting/augmentation has been explored before, it is challenging to judge what the specific novelty is.= beyond the new data reward *function* itself. The experiments are thorough and support the advantages of the algorithm. Table 1 and Table 2 show the performance of the method in the augmentation and the weighting conditions, but not when *both* are applied simultaneously. Next, the method is not tested for augmentation over the class imbalanced CIFAR dataset. The main drawback of the method is that the training data cannot be weighted and augmented simultaneously. Given the algorithm is general enough and based on the same data reward function, combining these two popular data manipulation strategies would be interesting.

Originality: The proposed framework is fairly novel and provides an interesting perspective on learning data manipulation. I found the Quality: The experiments on the text classification show that the proposed algorithms work well. However I found the experiments on image classification setting to be not very convincing~(see Improvements section) Clarity: The paper is well written and organized and contains sufficient details to enable reproducing the results. Significance: The proposed algorithm is flexible to incorporate different data manipulation schemes and provides a method to learn them to improve the end-task. This might enable integrating data generation methods~(GANs, VAEs) and learning an effective task-specific data-augmentation algorithms. ------------------------------------------- Post Rebuttal --------------------------------------- Having read the other reviews and the authors rebuttal, I stick to my initial evaluation that this is a good paper and should be accepted. The rebuttal provides satisfactory answers to my questions.