Part of Advances in Neural Information Processing Systems 5 (NIPS 1992)
Richard Zemel, Christopher Williams, Michael C. Mozer
We present a general formulation for a network of stochastic di(cid:173) rectional units. This formulation is an extension of the Boltzmann machine in which the units are not binary, but take on values in a cyclic range, between 0 and 271' radians. The state of each unit in a Directional-Unit Boltzmann Machine (DUBM) is described by a complex variable, where the phase component specifies a direction; the weights are also complex variables. We associate a quadratic energy function, and corresponding probability, with each DUBM configuration. The conditional distribution of a unit's stochastic state is a circular version of the Gaussian probability distribution, known as the von Mises distribution. In a mean-field approxima(cid:173) tion to a stochastic DUBM, the phase component of a unit's state represents its mean direction, and the magnitude component spec(cid:173) ifies the degree of certainty associated with this direction. This combination of a value and a certainty provides additional repre(cid:173) sentational power in a unit. We describe a learning algorithm and simulations that demonstrate a mean-field DUBM'S ability to learn interesting mappings.
Many kinds of information can naturally be represented in terms of angular, or directional, variables. A circular range forms a suitable representation for explicitly directional information, such as wind direction, as well as for information where the underlying range is periodic, such as days of the week or months of the year. In computer vision, tangent fields and optic flow fields are represented as fields of oriented line segments, each of which can be described by a magnitude and direction. Directions can also be used to represent a set of symbolic labels, e.g., object label A at 0, and object label B at 71'/2 radians. We discuss below some advantages of representing symbolic labels with directional units.