This paper proposes a strategy for an efficient deep network architecture search (here for image classification, but a the general idea would apply for other tasks as well). The proposed strategy is will motivated and involves a data sampling stage at each step. Here, an active querying strategy can be employed and the authors evaluate their strategy with three different active sampling strategies. They show that their strategy improves over active learning (with the same active query strategies) with a fixed architecture. However, the reviewers have rightly pointed out that a comparison with other architecture search strategies would also have been in place. The work in this submission is solid, and may become a useful reference for other research groups or practitioners. (As a side comment: "Deep Neural Architecture Search with Active Learning" may have been a more suitable title.)