This paper proves fast convergence of natural gradient descent for over-parameterized neural networks, and its generalization error bound. This paper is on the borderline and was carefully discussed. The main concern is about the novelty of this paper, as well as lack of details in the experiments. The paper gathered some support from the reviewers to merit acceptance, after author response and reviewer discussion. Thus I recommend accept.