The paper studies how task-relevant and task-irrelevant features are used to make predictions on artificial datasets (Navon, Trifeature) where the relationship between input features (color, shape, texture) and labels can be controlled for. The paper has a few interesting insights that are useful to understanding deep nets (e.g. when multiple features predict the label, models prefer easily-decodable features). Two main concerns raised by the reviewers (and left unaddressed in this work) are (1) whether the results here transfer to the real-world settings e.g. ImageNet and (2) the difficulty of features is not controlled. Overall, this is a good paper but definitely with rooms for improvement.