Summary and Contributions: This paper addresses the problem of causal inference when the true confounder value is a function of the observed non-outcome variables. The authors mention that under this setting, the positivity assumption i s violated; which as a result, causal inference is impossible in general. The contribution of this work is investigating two scenarios in which the causal effects are estimable.
Strengths: Causal inference is quite relevant to the NeurIPS community.
Weaknesses: Did not understand well enough to comment.
Correctness: Unfortunately, I could not understand the central claim of this paper: that if confounders are a function of the non-outcome variables, then positivity is violated in general. The authors explain in lines 92-93 that positivity does not hold when z is not equal to h(t). However, this never happens because z=h(t) by definition. I am confused ...
Clarity: I am a researcher working in causal inference, so I’m familiar with many concepts that the paper discusses. Yet, the paper was too difficult for me to understand. Too many inline math often interrupted the flow. Additionally, I think the readability of the submission could be improved by having a running example throughout the manuscript. Please consider re-writing your paper.
Relation to Prior Work: Yes, the prior work and their relationship to the current work is clearly discussed in the related work section.
Reproducibility: No
Additional Feedback: Please provide a response to my comment in the “correctness” section. ===== after rebuttal ===== I appreciate the authors's explanation on why the positivity is violated and that they have added this to their paper. However, I still think that the text is too hard to follow for a general NeurIPS audience though; which is why I have updated my score to "above threshold".
Summary and Contributions: This paper proposes a novel confounder model where the value of the confounder can be represented as a function of all non-outcome variables. - By investigating functional interventions, authors developed a sufficient condition to estimate their effects. - Considering intervening on all non-outcome variables, authors developed a sufficient condition to estimate the effect of the full intervention.
Strengths: - The theoretical model is well motivated by a concrete example. - Theoretical results are clearly presented and justified.
Weaknesses: - Not enough explanation on the theoretical assumptions. - No related work compared in the empirical evaluation.
Correctness: The logic is clear and makes sense to me
Clarity: This paper is well written.
Relation to Prior Work: Not clearly discussed.
Reproducibility: Yes
Additional Feedback: - The main results (Theorem 1 and 2) are based on several assumptions. It would be better to explain them briefly to give readers ideas about how strong they are and when they would hold. - This paper does not mention any other confounding models. I was wondering whether there exist similar works? If yes, it would be better to compare them in the experiments or explain why they were not compared. - The current manuscript only mentioned related work in the field of the motivating example (i.e., GWAS). It would be better to also review some similar confounding causal models in the introduction.
Summary and Contributions: The authors present a way to estimate causal effects when there is confounding present that can be expressed as a function of the observed variables. This type of confounding typically violates the positivity assumption and is present in e.g. GWAS studies. They establish a sufficient condition (F-positivity) for so-called functional interventions in this setting, as well as a necessary condition (C-redundancy) for non-parametric estimation of the full intervention. Finally they introduce and implement a method called LODE (Level-set order dependent estimation) that uses so-called surrogate interventions that can be estimated and match a conditional effect of interest, thereby effectively adjusting for the confounding. The effectivity of the approach is demonstrated on synthetic data as well as a real-world GWAS example on Celiac disease.
Strengths: Very good paper on an importnat and challenging, albeit somewhat niche, causal inference problem that occurs regularly in the context of GWAS analysis. Good mix of conceptual abstraction, solid mathematical rigour and technical implementation, with some interesting necessary and sufficient conditions, as well as highly nontrivial bounds on resulting conditional effect estimates.
Weaknesses: Its strength is perhaps also its weakness: the approach is technically and conceptually challenging, and covers a very specific causal inference problem requiring very specific case knowledge. To be precise: for when we want to estimate a causal effect, but we know that there is still a confounding component, but we also know that there is a functional/deterministic relation between that confounding component and other observable variables, and we also know the functional form of that relation, and we even know that this is also the only confounding component that remains. That is very specific, however, it happens to be a reasonable situation in GWAS analysis, and so it is more than just an exotic theoretical tour de force. But even then I doubt that many practitioners will be able to easily apply the method, and so there is still a danger that in practice the method becomes a case of a solution that was too difficult for the problem.
Correctness: As far as I can tell the claims and results are correct, although I have to admit that I could not follow all steps in detail, and only skimmed the proofs in the supplement. Experimental evaluation is relevant and to the point. Only caveat is the closing claim in the GWAS example in 4.2 ‘Thus LODE adjusts for confounding factors that the outcome model ignored’: I think they only show that we do indeed get a different result, but whether that actually corresponds to ‘correctly adjusting for confounding factors’ has not been proven. Although I am happy to give the authors the benefit of the doubt here.
Clarity: The paper is well-written, but despite the clearly best efforts of the authors it remains highly abstract and technical, which makes it a tough read for anyone not very familiar with the problem already. I guess that this is unavoidable given the material involved, although the paper could benefit from a few more concrete examples. (in particular for the 'surrogate interventions')
Relation to Prior Work: Yes: current work is clearly novel.
Reproducibility: Yes
Additional Feedback: In short: good paper on difficult problem. Quite challenging, but clearly deserves to be in the conference. => accept minor comment: l.65, l.91: typo PCA ’principle’ => ‘principal’