This paper proposes a "twist" in the usual alternating direction method for CP, and proves results that some reviewers found surprising (in a good way). As R3, said, it offers a rare combination of theoretical guarantees and practical improvement. R4 brought up some issues, most notably with the assumption of the tensor rank not being too large, as well as concerns about the experiments and not providing code. Several reviewers wanted to see comparisons with more methods (beyond ALS). There was also an overall concern that with 30 pages of supplement, it was unrealistic for reviewers to be expected to check all details of all proofs. In the AC's own view, these kinds of submissions are not appropriate for NeurIPS, where reviewers have very limited time and many papers to review. Due to the range of reviewer disagreements, we added an additional reviewer, who generally found the paper above-the-bar, but was worried the authors would completely ignore some of R4's comments. In particular, R5 wants (1) discussion about limitation to under-complete case and scaling to higher order, (2) comparisons beyond ALS, (3) add literature as pointed out by R4 and R3, and (4) fix tables as pointed out by R3. Overall, we feel that this is a paper that could potentially have a large impact, and therefore recommend accept.