The paper proposes a method for pruning deep networks based on the largest values of the gradient vector. The idea is new compared to previous attempts; although it is somewhat related to Fisher pruning, that is also based on magnitudes of gradients, the method here is more of an SGD variant rather than a post-training evaluation method. The techniques do not come with rigorous guarantees, but the reviewers agree that the experiments and surrounding studies are interesting enough to incite future research around this method. Based on this, I believe the paper would be a appreciated by the attendees of NeurIPS The idea is to update only the values having the maximum gradient magnitude Though the results are somewhat exploratory and preliminary, their potential is great and their novelty is significant enough to be presented in NeurIPS