The paper has three interesting contributions: Theoretical contributions: The paper improved the upper bound over the number of communication rounds for non-convex optimization problems under Polyak-Lojasiewicz conditions. While the extension from strong convexity to PL is somewhat expected, the paper is clearly written and handles nicely the "relaxing bounded gradients" assumption. Algorithmic contributions: An adaptive scheme for choosing the communication frequency has been developed. One reviewer raised that the adaptive scheme should have been developed with more details, and that could have been an even more significant contribution (new algorithm). Empirical contributions: Experimental results on Amazon EC2 cluster and an internal GPUs cluster is definitely a plus