This work provides an interesting approach for continual learning over time that is able to work in both task-specific and task-agnostic scenarios. The paper is well written and easy to read. The author response addressed some of the reviewer's concerns and led to the decision to accept assuming the comparisons with Bayesian continual learning and same-size architectures are included in the final paper. The work could be improved by adding more heterogenous and complex tasks or by extending to domains outside of vision.