Summary and Contributions: Combines compartmental disease modeling with machine learning for COVID-19 forecasting. It surpasses state-of-the-art methods by some well motivated additions to existing models. It also incorporates a level of interpretability. Contributions include an extension to the standard SEIR model with additional compartments, time-varying encoding of the covariates, and learning mechanisms to improve generalization while learning from limited training data.
Strengths: Systematically integrated additional covariates into the compartmental model of SEIR. Allows covariates to be time-varying. Appendix is very detailed explaining variables and each data source so the results can be reproduced. Performance exceeds all baselines. The paper also includes county level forecasts. The additional results provided in the appendix further support the paper's claims. The demonstrated explainable insights also provide support to the interpretability claim.
Weaknesses: Does not provide code and a lot of relevant information is pushed to appendix. Most of this information is required for reproducing the results. Given the constraints of paper length, I consider this acceptable. Line 127: Table 2 does not contain the variables mentioned here. I believe this refers to Table 2 in Appendix?
Correctness: Yes both are correct. The method is developed incrementally and is easy to follow.
Clarity: Yes
Relation to Prior Work: Yes, it discusses prior work in good detail and clearly shows how it defers from them.
Reproducibility: Yes
Additional Feedback: Overall, there are enough contributions and improvement in results from existing approaches. The interpretability claim is also justified through multiple demonstrations. The appendix is very detailed and when seen in conjunction with the paper, it is easy to follow the method. All data is pulled from open sources. Would still prefer the code to be released. Having read other papers of COVID-19 forecasting, this paper appears to me as well thought out with enough important contributions to be very useful. Finally, the authors feedback was great in addressing comments.
Summary and Contributions: This paper proposes a novel approach that modeling COVID-19 progression. The authors extend the standard SEIR model with a newly designed compartment for undocumented cases and hospital resource usage and also incorporate understandable co-variates such as mobility indices into the model. During training, the authors combine several techniques to overcome the overfitting problem and improve generalization. Using the current public US COVID-19 dataset, authors compared the performance with a number of current public benchmark approaches.
Strengths: Compare to other epidemic forecasting, the major challenge in current COVID-19 pandemic forecasting is that there are many potential sources of data, but their causal impact on the disease is unclear and the progression of the disease influences will largely influence the public policy and individuals’ public behaviors and vice versa. It's important to extract useful features from these covariates and design a proper system to incorporate them into the model. Thus, the major contribution of this paper can be summarized as: 1. The proposed compartmental model is novel and carefully tailored for COIVD-19. Based on the standard SEIR model, authors introduce compartments such as undocumented infected and recovered cases, hospitalized/ICU and ventilator to the model. They also give proper assumptions such as partial immunity based on the latest medical research. 2. Authors select many important covariates that could have an impact on the model compartments such as mobility, hospital Resource availability, and use an encoder to incorporate these covariates into the model. In addition, in this design, the proposed model can provide explanatory insights, for example, the mobility index has a contribution to the infectious rate while the school close has a negative weight.
Weaknesses: 1. Related Work. In the related work section, the authors summarize some related models for infectious diseases and address their weaknesses. However, it seems none of them are used as baseline models for comparison in the experiment section. Instead, the authors present the results of five top-performing models designed for COVID-19. It would be better if the authors can give a summary of these COVID-19 models to address their weakness and point out the major improvement of their method compared with these ones. 2. Experiment: a. Part of the ablation studies is unclear to me. The author claims the extra compartment has significant benefits for the prediction. However, Table 4 does not directly show this benefit. If we compare the first row and second row, the covariates encoder seems to offer the largest benefit, but the author does not mention this. It would be better if the author makes a clearer table, and show different results based on a different combination of the techniques they applied. b. The author compares their model with 5 top-performance COVID-19 forecasting models and claims their method outperforms the next best model by a large margin. However, in Table 3 in Appendix H., when the prediction horizon is 5 days, although the author bolds their results, their model cannot beat YYG in 3/5 dates. In addition, the comparison is unfair, the proposed model uses an additional 16 covariates data, while others are different. I would suggest the author provide a deeper discussion between their model and others instead of citing some number, especially the YYG model which is also based on the SEIR and have close performance.
Correctness: As mentioned before, the comparison seems unfair, the author would justify, even with additional encoder of co-variate data, why their model can not beat YYG when prediction horizon is small.
Clarity: The paper is clearly written and not hard to follow. However as stated before, a proper related work and more insight discussion are desirable.
Relation to Prior Work: Not completely, as mentioned before, I would discuss the major differences between their models and other COIVD-19 forecasting methods they compared in the experiment part.
Reproducibility: No
Additional Feedback: By updating "related work", adding new "ablation study" and ''model comparison" sections, authors have addressed most of my concerns satisfactorily. Thus I would like to increase my overall score and recommend acceptance of this paper.
Summary and Contributions: The paper proposes a new compartmental model by improving an well-established model for contagious viral diseases, accounting for undocumented cases, hospitalization, icu stays and necessity for ventilator use.
Strengths: This work improved an well-established model to incorporate factors that were identified as important in the current global pandemic. COVID-19 is currently the most relevant topic globally and improving upon a traditional model and also providing tools that utilize other techniques to complement a compartmental model fits the purpose of the conference very well.
Weaknesses: The authors did elaborate well on how their model could be actually used by policy makers. The work would be highly enriched if they introduced why their choices improve decision making as a reasoning behind their modeling.
Correctness: There were no identified errors in their claims and methodologies.
Clarity: The paper is well written and all the methods all well described. The Supplemental material is valuable and provides useful insights into their development methods.
Relation to Prior Work: The authors discussed previous work as well as provided clarity as to how their work differed from the other similar models.
Reproducibility: No
Additional Feedback: - The development of forecasting tools for covid-19 has been a popular topic. The authors should tone down their claims that their model outperforms the next best model as things are changing quickly and literature on this has vastly increased over short periods. - The use of static rates is the reason SEIR has been popular among policy makers. For example, it allows them to make quickly change the social distancing measures based on the trend of the forecast given a mobility rate. The new compartments added by the authors can allow them to make decisions regarding other aspects such as availability and relocation of ICU beds and ventilators. While introducing a trainable model to create time-sensitive variables does give insight to what covariates might be affecting the current trends, it will limit the capability to simulate other scenarios as the model will fail to incorporate major changes in policy, which is partially due to collinearity and also due to the fact that their model does not account for how much each timestep influences the prediction interval. This could be observed in the paper, but the authors did not state this as a limitation. Minor fixes: - There is a typo in the figure 2, the recovery rate for the icu recovery p(c) is written as p(u).