*PROS: provides an understanding the relationship between deep ensembles and posterior inference, several experiments that demonstrate the usefulness of the proposed method, including additional ones in the rebuttal. *CONS: high computational overhead. R3 makes several good comments that should be addressed in the final version Meta-reviewre recommendations: The paper is boerderline but I recommend acceptance for the reasons mentioned by R2 in his post rebuttal update. In particular, that this work presents a formal argument, based on recent work on neural tangent kernel (NTK), that standard deep ensembles do not have valid interpretations as Bayesian posterior predictive approximations. I recommend the authors to take into account the reviewers points to improve the paper for the final version.