NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:3187
Title:Scalable inference of topic evolution via models for latent geometric structures

Reviewer 1

The authors propose a set of models and inference algorithms to model the evolution of topics over time. The proposed set of models are novel in terms of both the generative models and inference techniques. The novelty in generative models is achieved by representing a set of topics at a given time as a topic polytope and modeling evolution of topics as trajectories in the geometric space of the polytopes. The proposed inference approach is fast and scalable, and lends itself well to online learning and distributed computing. The authors describe the motivation for the generative process and the algorithms in sufficient detail. The supplement further elaborates on inference details and sensitivity to priors. Experimental results demonstrate improvements in perplexity and significant speed ups in run time. The major contributions are clear and significantly advance the state of research.

Reviewer 2

Originality: This is a nice combination using the vMF model and Hungarian matching. I really like the ideas here and the performance is great Quality and Clarity: The basics are ok: citing related work, using those ideas to make your point, etc. But I feel that you miss some opportunities to make the work more accessible. You gave Algorithm 1, but it is hardly worth it, I needed more detail. Also, if you are going to give an algorithm block, why not for SDDM? Please clearly link the mathematical ideas to the algorithm? Likewise, the math jumps in hard to follow ways. You may laugh but I am not sure where the cost (164, 214 and 244) are coming from and how the ideas are represented in those formulas. The body of the paper reference nice mathematical ideas but I found it difficult to get the ideas. Significance: as I said above, the speed of this approach seems very valuable.

Reviewer 3

This is a very well written paper, both in style and substance. There are a few stylistic peculiarities that could surely be ruled out by thorough proof-reading. The authors present a nice introduction into the idea of modelling sets of topics, i.e. sets of points on a simplex, as the geometric structure of a polytope. They go on to describe, how evolution of such a polytope can be modelled over time by embedding a unit hypersphere into the simplex and modelling polytope evolution as random trajectories over this sphere. They further present a non-parametric hierarchical model for capturing polytopes with a varying number of topics and also multiple polytopes arising from different corpora. Their experimental section deals with two different data sets, a medium sized one (400k documents) and a large one (3M documents). While the sizes of the corpora are appropriate to demonstrate the performance of the inference algorithm, usage of more well-known corpora such as the NYT corpus would have been beneficial in terms of comparability to previous approaches to the problem. Also, vocabulary truncation to just over 4500 terms from 400k documents and 7300 words from 3M documents seems rather aggressive and needs further elaboration. Although there is a case study for a certain topic, including probability trajectories for its top words, a more extensive qualitative assessment of model performance would be beneficial. Since topic models are primarily successful because of their interpretability by humans, it is often useful to demonstrate qualitative over quantitative results.