Summary and Contributions: Derives a biologically plausible version of SFA, and runs it on a standard moving image datasets, obtaining results similar to previous batch learning methods.
Strengths: An elegant formulation of the problem and nice solution.
Weaknesses: Perhaps could benefit from stronger ties to potential neural mechanisms, or where in the brain you may expect to find this, experimental predictions, etc. It seems this is the main point after all. Also, how do you propose that biology computes the nonlinear expansion \mathbf{x}_t?
Correctness: yes as far as I can tell
Clarity: Yes.
Relation to Prior Work: The development in section 2.1 and 2.2 seems straight of Wiskott's papers, but you wouldn't really know that from reading the text. Its cited by numbers, but I think highlighting more directly in the text would be helpful.
Reproducibility: Yes
Additional Feedback: I find the paper interesting, but somehow not super interesting. It's never seemed an issue to me that you could have a biologically plausible version of SFA. Foldiak's original model was formulated that way after all. So not sure what this development adds in that context. I'm sure there advantages to this approach, but it would be good to bring them out more.
Summary and Contributions: The paper develops a slow feature analysis (SFA) method that is biologically plausible, in that it operates online and has weight updates that only use information that is available in the presynaptic and postsynaptic neurons. The method is tested on several datasets, and it is shown that cost is reduced with training and that interesting receptive fields emerge.
Strengths: - I agree with the authors that a biologically plausible implementation of SFA is an important development. - The development of the method from standard SFA was systematic and rigorous.
Weaknesses: - Some limitations were mentioned in the discussion. Of these, the fact that the method uses linear neurons seems to be the most serious. I appreciate that the authors defined their sense of "biologically plausible", but linear neurons don't sit very well with this term.
Correctness: Yes, it seems to be correct.
Clarity: The writing is clear and well organized. The math was clearly and efficiently communicated.
Relation to Prior Work: Yes.
Reproducibility: Yes
Additional Feedback: - I had regrettably not checked many of the equations in the first round, but I have checked most of them now. - Thank you for addressing my minor concern about the assumption of full rank in Line 108. I look forward to the updated explanation in the final version of the paper.
Summary and Contributions: This paper produces a so-called biologically plausible neural network for slow-feature analysis. Biological plausibility here means that network learning is online and based on local synaptic learning rules. These online and locality requirements might lead to low computational overhead. While Foldiak, Wiskott, and many others have explored online local learning for SFA in the last thirty years, this paper attempts to relate SFA to a normative theory through an MDS objective.
Strengths: While Foldiak, Wiskott, and others have explored local online learning for SFA in the last thirty years, this paper successfully relates it to the normative MDS objective. Similar work on biologically plausible implementations has been one-dimensional; this work extends that to multiple dimensions.
Weaknesses: The conceptual and theoretical innovations are limited, which is not surprising given that the problem has been worked on for the last thirty years, most notably by Wiskott's lab. Claim of biological plausibility seems weak, limited only to local learning rules and online learning.
Correctness: The basic idea and the logic seem to be sound, though I have not scrutinized all the equations.
Clarity: Reasonably clear and well written.
Relation to Prior Work: This work seems to be a marriage of Wiskott's line of research and Pehlevan and Chklovskii's recent approach.
Reproducibility: Yes
Additional Feedback: I read all the reviews and rebuttal, and can see the potential contribution relative to existing work better. I keep my score. It is an acceptable paper.
Summary and Contributions: While SFA is a classical and widely used unsupervised learning algorithm, a neural plausible neural algorithm for SFA is still missing. This paper presents a solution and the numerical results well support the proposed method. So overall, it's a very useful piece of work.
Strengths: 1. The presented method is very useful to the computational neuroscience community. 2. The paper presents a reformulation of SFA, which leads to a neural plausible SFA algorithm. 3. The numerical results seem to be solid and well support the proposed theory.
Weaknesses: There are some minor typos and notation issues, just like most of the NeurIPS drafts. If accepted for publication, the authors should definitely make an effort to polish the paper.
Correctness: I checked most of the derivation and I think they are correct. Except there are a few minor typos, e.g. the equation above equation (7), V and V^T are missing. The optimization above equation (8), a transpose is missing. Things like these should be fixed but do not bother me much.
Clarity: Section 2 is too long, some of the unnecessary review shall be moved to the appendix. Section 3 shall be expanded and discuss the major results more thoroughly.
Relation to Prior Work: This piece of works has been missing for a long time. I think it can fill the gap.
Reproducibility: Yes
Additional Feedback: