All reviewers were positive about the paper. The paper corrects several common incorrect assertions and misleading derivations in the natural gradient algorithms literature. The exposition is remarkably clear, with a potential to serve as a reference paper on the topic. The paper is clearly of broad interest to the machine learning community. We recommend to take the reviewers' comments and suggestions into account while preparing the camera ready final version of the paper. The authors might also want to consider an alternative title that would better reflect the scope of the paper and in particular better support its potential to be a reference paper for researchers interested in natural gradient algorithms. Accept.