This paper provides evidence that the number of possible activations is quite small, in stylized settings (it seems the paper is based mainly on remark 1). The paper is interesting and appears to be the first most concrete evidence of this, though to some extent this is implied by ntk analyses; I ask that the authors provide a concrete technical comparison to ntk and related papers/consequences. I also ask for a more thorough comparison to VC dimension results, which also proceed via region (and activation) counting.