Part of Advances in Neural Information Processing Systems 14 (NIPS 2001)
Alberto Paccanaro, Geoffrey E. Hinton
We present Linear Relational Embedding (LRE), a new method of learn- ing a distributed representation of concepts from data consisting of in- stances of relations between given concepts. Its final goal is to be able to generalize, i.e. infer new instances of these relations among the con- cepts. On a task involving family relationships we show that LRE can generalize better than any previously published method. We then show how LRE can be used effectively to find compact distributed representa- tions for variable-sized recursive data structures, such as trees and lists.
1 Linear Relational Embedding
Our aim is to take a large set of facts about a domain expressed as tuples of arbitrary sym- bols in a simple and rigid syntactic format and to be able to infer other “common-sense” facts without having any prior knowledge about the domain. Let us imagine a situation in which we have a set of concepts and a set of relations among these concepts, and that our data consists of few instances of these relations that hold among the concepts. We want to be able to infer other instances of these relations. For example, if the concepts are the people in a certain family, the relations are kinship relations, and we are given the facts ”Alberto has-father Pietro” and ”Pietro has-brother Giovanni”, we would like to be able to infer ”Alberto has-uncle Giovanni”. Our approach is to learn appropriate distributed rep- resentations of the entities in the data, and then exploit the generalization properties of the distributed representations [2] to make the inferences. In this paper we present a method, which we have called Linear Relational Embedding (LRE), which learns a distributed rep- resentation for the concepts by embedding them in a space where the relations between concepts are linear transformations of their distributed representations.
involve two concepts. Let us consider the case in which all the relations are binary, i.e. , and the problem In this case our data consists of triplets we are trying to solve is to infer missing triplets when we are given only few of them. Inferring a triplet is equivalent to being able to complete it, that is to come up with one of its elements, given the other two. Here we shall always try to complete the third element of the triplets 1. LRE will then represent each concept in the data as a learned vector in a