This paper studies continual learning that does not require task boundary and identity information and proposes a novel model ensemble method from the combinatorial perspective for this problem. All reviewers and AC agree that this paper builds a novel and promising direction. Authors also design delicate algorithm by introducing the non-stationary learning techniques to solve this problem. The experimental results of this method are somewhat weak in several aspects, but given the challenge of online continual learning in nature, they are fairly convincing to justify the main ideas and proposed methods. Note that after rebuttal and discussion phases, there still remain several major concerns: First, the empirical evaluation is not realistic in terms of task diversity and scalability. Second, the explanation of the method still lacks clarity and the paper is not easy to follow. In particular, there is not a clear reason the node level averaging should be meaningful. Since the rebuttal only partly addresses the above comments, the authors are required to improve their paper dedicatedly following these comments. I recommend to accept this paper, assuming that the above major concerns will be addressed fully in the camera-ready version.