NeurIPS 2020

Duality-Induced Regularizer for Tensor Factorization Based Knowledge Graph Completion

Meta Review

There are roughly two different approaches in the literature for knowledge graph completion (KGC), namely distance based (DB) models and tensor factorization based (TFB) models. Although both approaches have their own advantages and disadvantages over each other, TFB models cannot attain state-of-the-art performance due to overfitting problem, and therefore various regularizers are employed for TFB models. In the paper, authors propose a regularizer for TFB models, namely Duality-induced Regularization (DURA), which is inspired by the score functions of the DB models. They come up with a dual problem which involves a distance based KGC model, and show that when the aforementioned regularizer is employed for the primal problem (i.e. TFB model), both problems become equivalent. By doing so, they are able to shed light on the connections between TFB and DB models, as well as attaining a much better performance on TFB models. The rebuttal has been found quite useful by the reviewers to resolve issues. The experimental results are also found convincing. The paper has several strong points - Well written and the main idea of the paper is easy to grasp and all the related technical concepts are explained in a clear way. - Empirical evaluation shows that the proposed regularization methodology significantly increases the performances of the tensor factorization based models. When compared to some other regularization methods, DURA performs still better in general and it finds sparser embeddings for the entities. - Proposed regularization for tensor factorization based models is found in a very intuitive way, i.e. by introducing a distance based dual model. Thereby, it establishes an implicit connection between tensor based models and distance based models. - The implications of why employing DURA may help to prevent overfitting is discussed in great detail. - When the relation embedding matrices are diagonal, DURA gives an upper bound tensor nuclear 2-norm of the observed binary tensor, which in turn provides further implications about how DURA may help tensor factorization based knowledge graph completion. A number of weaknesses have also been mentioned but the authors have successfully answered many of these.