This paper proposes the use of Gaussian process factor analysis (GPFA) to analyze the latent dynamics of spontaneously generated electrical activity of the brain as measured using fMRI. The paper shows that GPFA can be used to extract slowly varying latent factors that are informative for understanding cognitive processes in healthy brains and cognitive decline in diseased brains. While GPFA is not a new method and has been applied in many other areas of data analysis, it has not previously been applied to the problem of inferring latent dynamics from fMRI data. Further, as the authors show, this approach appears to yield superior performance to a range of alternative approaches on two different tasks using large, real data sets. The reviewers judged the empirical results to be highly significant as they point to new data-driven ways of understanding the relationship between brain and behavior. All reviewers were satisfied with the author response following the discussion of the paper and the consensus of the reviewers is that the paper should be accepted. The authors should be sure to incorporate the responses to the reviewers original concerns into the updated manuscript and to address any issues that remain in the final reviews.