Dear Authors, Thank you for your submission to NeurIPS. Your work has generated fruitful and interesting discussions, where both pros and cons about your proposal were raised. The reviewers consent for acceptance. Two points to be considered before the final version of the paper: 1. Please elaborate more on Line 54 implies elimination of the condition of sum of squares of step sizes less than infinity. 2. In the conclusions section, it is clear the statement for co-ordinate descent is true; the proof for co-ordinate descent is still an open question. We encourage the authors to restate the conclusions, as reviewers are concerned that the proof breaks down for any first order method which has noisy estimate of gradients like block co-ordinate descent, SGD, etc.