The contribution of this paper resides in the combination of known techniques (BO, Thompson sampling and regret minimization) to tackle a practical problem that had modest attention in the literature. The paper also provides theoretical guarantees on minimizing overall regret. I would encourage the authors to consider the outstanding concerns of the reviewers and to include the suggested round-robin baseline.