The paper introduces gossip based algorithms for consensus and community managing asynchronous updates in distributed Deep RL. The reviewers had some concerns regarding the comparison of the proposed approach to baselines (and the choice thereof), but were overall impressed by the empirical results, that show a more computational efficient algorithm (compared to A2C/A3C) with no significant loss in learning performance. Please take into account the detailed comments of the reviewers (especially R2 and R3) when preparing the final version.